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ABSTRACT 


The implementation of quality standards within the 
European Community by the creation of International Quality 
Standards 9000 is another step toward development of a global 
marketplace. It is in the interests of DoD to support this 
trend in order to help maintain the defense industrial base. 

The first part of this study performs a comparison of DoD 
quality standards to the ISO 9000 Standards. The second part 
of the study consists of a survey of U.S. firms that have 
become ISO 9000 registered. This survey is intended to 
provide an assessment of the current movement within the 
defense industrial base toward adoption of ISO 9000 Standards. 
The survey also attempts to identify potential implementation 
issues relating to adoption of ISO 9000 Standards in place of 
military standards. It is concluded that DoD should implement 
ISO 9000 and that the impact of this implementation will be 
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I. PURPOSE/INTRODUCTION 


A. GENERAL INFORMATION 

The primary standards cited in Department of Defense (DoD) 
contracts for quality assurance programs are Military 
Specifications MIL-Q-9858A (MIL~Q) and MIL-I-45208A (MIL-I). 

The emergence of a global marketplace has led to the 
creation of quality control standards that will apply to this 
new marketplace. The International Standards Organization 
(ISO) has developed the ISO 9000 Quality Management and 
Quality Assurance Standards(ISO 9000). These new 
international standards are applicable to countries in Europe 
that will form the European Community (EC) in 1992. 

B. OBJECTIVE OF THE RESEARCH 

The purpose of this study is to determine if DoD should 
enact the implementation of ISO 9000 standards in weapon 
systems contracts. The primary research question is: Should 
ISO 9000 be implemented within DoD and the Defense Industrial 
Base, and when implemented how will it affect DoD contracting? 
In addition to the primary research question, the following 
subsidiary research questions will also be addressed: 

1. What is ISO 9000? 
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2. What are the similarities and differences between ISO 
9000 standards and Military Standards (MIL-Q-9858A and MIL- 
1-45208)? 

3. What is the policy of DoD with regards to ISO 9000? 

4. Of companies within the Defense Industrial Base that have 
completed ISO 9000 certification, what are the anticipated 
benefits? 

5. Should ISO 9000 be recommended for implementation within 
DoD and the Defense Industrial Base, and if so, what action 
must be taken to accomplish this objective? 

C. SCOPE AND LIMITATIONS OF THE STUDY 

This study involves a detailed assessment of ISO 9000 
standards as compared to the current Military Standards. The 
study involves gathering information from Government and 
commercial organizations to determine the current state of 
knowledge as well as institutional forces that either support 
or reject the implementation of ISO 9000 in place of Military 
Standards. The study is not limited as to size of defense 
contractors, or to a specific type of industry; suggested 
recommendations for implementation are applicable to the 
entire Defense Industrial Base. 

D. METHODOLOGY 

The research data were collected through an extensive 
literature search and by telephone interviews. 

The literature search, comprising professional journal 
articles and current regulations and directives, was made 
through the Naval Postgraduate School Library, the Defense 
Logistics Studies Information Exchange (DLSIE), attendance at 
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an ISO 9000 workshop sponsored by the University of Wisconsin- 
Whitewater, and from several DoD offices responsible for 
Quality Assurance. This survey provides a background of 
information that defines and describes both ISO 9000 standards 
and MIL-STDS that apply to quality assurance systems. 

The researcher completed 25 telephone interviews with 
managers and directors of contractor quality assurance 
programs. Appendix A provides a listing of the companies 
contacted. The interviews were all held on a nonattribution 
basis in order to obtain candid responses and honest 
evaluations of current and proposed implementation plans for 
ISO 9000 standards. The interviews were conducted by 
telephone to allow the researcher to obtain and explore 
information from people who had extensive experience in 
implementing ISO 9000 standards. 

E. ORGANIZATION OF THE STUDY 

Chapter II provides background information and 
descriptions of ISO 9000 standards and Military Quality 
Assurance standards. 

Chapter III compares the underlying philosophies of ISO 
9000 to Military Standards and draws detailed comparisons of 
the two sets of standards. 

Chapter IV presents data and a description of methods used 
to collect the information. This chapter also includes an 
analysis and interpretation of the data. 
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Chapter V contains conclusions and recommendations 
regarding the implementation of ISO 9000 as well as areas for 
further research. While answering research questions, this 
final chapter also discusses the conclusions based upon 
research data, makes specific recommendations resulting from 
the research effort, and suggests possible areas for further 
research. 
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II. BACKGROUND 


A. INTRODUCTION 

The decline in DoD procurement dollars has led many 
companies to search for new markets overseas. As this 
expansion into the international market occurs, it is logical 
that DoD contractors will argue for increased standardization 
between current United States standards and international 
standards used in contracting. 

One of the primary areas of interest for standardization 
lies in Quality Assurance. The area of quality control should 
be one in which the buyer and seller agree on an acceptable 
method, since standardization of quality assurance systems 
across international boundaries has the potential of 
significantly reducing quality inspection costs. These 
standardization agreements create special problems but hold 
great potential when viewed from the standpoint of 
international trade. In order to achieve the desired quality 
of products between competing producers, adherence to 
international standards gives the purchaser an opportunity to 
reduce duplication of quality inspections without sacrificing 
the quality of the end product. Additionally, an 
international standard that is understood and acceptable to 
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the buyer reduces the complexity of assessing a potential 
supplier's quality assurance prograni. 

The creation of the European Economic Community in 1992 
has led to the formation of international quality standards 
(ISO 9000 standards). This standardization of quality 
assurance in Europe will certainly impact U.S, firms and the 
DoD industrial base. The acceptance of ISO 9000 standards for 
use in DoD contracts has already been proposed within DoD. 
The next step in implementing ISO 9000 becomes one of 
educating and familiarizing both industry and Government 
personnel in the use of these standards. 

The motivation to accept international standards is based 
on the need for market expansion. As companies expand their 
foreign sales, they will be forced to adopt ISO 9000 in order 
to remain competitive in overseas markets. The use of one 
quality standard for both DoD and international sales promises 
reduced administrative and maintenance costs relating to 
quality assurance programs. 

The next section of this study will provide a broad 
background in both ISO and MIL-STD Quality Assurance 
standards. 

B. ISO 9000 STANDARDS 

The ISO 9000 standards comprise five (5) separate 
documents. A listing, accompanied by a brief description, is 
as follows:[Ref l:p.23] 
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• ISO 8402. Quality, Vocabulary. This is simply a 
reference document that defines terms used in ISO 9000 
standards. 

• ISO 9001. Quality Systems. Model for quality assurance 
in design/development, production, installation and 
servicing. This is a specific model for companies that 
have all phases of the manufacturing process from design 
to final product, and is the most comprehensive of the ISO 
standards. 

• ISO 9002. Quality Systems. Model for quality assurance 
in production and installation. This is a specific model 
for manufacturing companies that have all phases of 
manufacture except design of the product. 

• ISO 9003. Quality Systems. Model for quality assurance 
in final inspection and test. This gives specifics for 
end-item inspection procedures. 

• ISO 9004. Quality Management and Quality System Elements 
Guidelines. This is another reference document that 
explains the philosophy and underlying purpose of ISO 
9000 . 


The ISO 9000 standards are based on an approach to quality 
assurance that models itself along Total Quality Management 
(TQM) principles. The aim of the standards was summarized by 
Trevor Davis from the quality management group at Coopers & 
Lybrand Deloitte, as follows:[Ref l:p.25] 


• to increase customer confidence in the company, by 
providing a common framework across Europe (and the 
world) ; 

• to move from a system of inspection to one of quality 
management; 

• to remove the need for multiple assessments of suppliers; 

• to gain management commitment;to link quality to cost 
effectiveness; and 

• to give customers what they have asked for. 
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The above summary highlights some of the main themes that 
run through the ISO 9000 standards. Many of the principles 
listed clearly reflect the TQM approach. 

Each of the ISO 9000 documents covers different phases of 
the quality assurance area, and each document is briefly 
described below. 

1. ISO 9004, Quality Meuiagement: Specification for 
Design/Development, Production, Installation and 
Servicing. 

The dociament that gives broad guidelines and amplifies 
the other four ISO documents is ISO 9004. The ISO 9000 
standards were written with two applications in mind. The 
first was the creation of guidance for companies to use in 
developing their quality assurance organizations; the second, 
a program that would satisfy the contractual quality 
requirements dictated by customers.[Ref 2:p.2] 

The ISO 9004 document provides guidance in creating an 
internal quality assurance organization. The document begins 
with a list of definitions which incorporates ISO 8402 
definitions into the ISO 9000 standards. ISO 9004 also 
outlines concepts regarding principles of Quality Assurance 
and the proper selection of ISO 9001, 9002 or 9003 for 

specific applications. ISO 9004 concludes by bridging the 
internal/external applications of the ISO 9000 standards. It 
clearly calls for the use of customized versions of the 9001, 
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9002, and 9003 documents. The ISO 9000 documents state that 
it is not the intention of ISO to force suppliers into the 
acceptance of one standardized quality assurance organization. 
Rather, the standards are meant to assure both the supplier 
and customer that a product/service is being provided that 
will satisfy contractual requirements for quality. 

2. ISO 9001, Quality Systexos: Specification for 

Design/Development, Production, installation and 
Servicing. 

This portion of ISO 9000 is intended "for use when 
conformance to specified requirements is to be assured by the 
supplier during several stages which may include design/ 
development, production, installation and servicing."[Ref 
4:p.l] Paragraph 4 of this portion of ISO 9000 gives an 
outline of a quality assurance program that takes the 
following form: 

• 4.1 Management responsibility 

• 4.2 Quality system 

• 4.3 Contract review 

• 4.4 Design control 

• 4.5 Document control 

• 4.6 Purchasing 

• 4.7 Purchaser supplied product 

• 4.8 Product identification and traceability 

• 4.9 Process control 
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• 4.10 Inspection and testing 

• 4.11 Inspection, measuring and test equipment 

• 4.12 Inspection and test status 

• 4.13 Control of nonconforming product 

• 4.14 Corrective action 

• 4.15 Handling, storage, packaging and delivery 

• 4.16 Quality records 

• 4.17 Internal quality aiidits 

• 4.18 Training 

• 4.19 Servicing 

• 4.20 Statistical techniques 

The ISO 9001 document is written in simple, easy-to- 
understand language, and many of the paragraphs listed above 
are subdivided into parts composed of one sentence. As an 
example, paragraph 4.15.2, "Handling:" 

The supplier shall provide methods and means of handling 

that prevent damage or deterioration.[Ref 3:p.6] 

Numerous other examples can be seen in Appendix B 
which provides a comparison of ISO 9001 to MIL-Q-9858A. It 
should be noted that Appendix B lists ISO 9001 paragraphs in 
the right-hand column of the appendix, and the entire appendix 
is structured using the format of ISO 9001. 

ISO 9001 is a model for a quality assurance 
organization that is engaged in design, development and 
production of a complex product requiring conformance to 
buyer-directed specifications. It does not give a micro- 
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management approach to the quality program being used by the 
supplier. Suppliers are free to implement the model in a way 
that best suits their production process. Yet the model does 
cover all the areas of a sound quality assurance program. 

ISO 9001 is meant to be a model, designed for systems 
procurements that include the design of the end product by the 
manufacturer. It covers all aspects of producing a complete 
weapon system as specified by a buyer. The supplier is 
expected to conform to buyer specifications in several phases 
of the procurement. The ISO 9002 standard takes the systems 
approach and reduces it to an established product line. 

3. ISO 9002, Quality Systems: Specification for 

Production and Installation. 

The functions of design/development and servicing have 
been removed from ISO 9001 in order to produce the ISO 9002 
document. This portion of ISO 9000 is intended "for use when 
conformance to specified requirements is to be assured by the 
supplier during production and installation."[Ref 4:1] The 
outline of the model in this document is composed along the 
same lines as shown in paragraph B above, with the removal of 
subparagraphs Design Control and Servicing. The ISO 9002 
document is very similar in detail to ISO 9001, except it 
assumes a mature product with a design not subject to much 
change. 
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The ISO 9003 document takes the ISO 9002 and reduces 


it even further to where it becomes appropriate for use in 
established production lines that are built to producer 
specifications. 

4. ISO 9003, Quality Systems: Specification for Final 
Inspection and Test. 

This portion of ISO 9000 is intended "for use when 
conformance to specified requirements is to be assured by the 
supplier solely at final inspection and test."[Ref 5:p.l] 
Paragraph 4 of ISO 9003 outlines a quality assurance program 


as 

follows: 

e 

4.1 

Management responsibility 

• 

4.2 

Quality system 

e 

4.3 

Document control 

• 

4.4 

Product identification 

# 

4.5 

Inspection and testing 

e 

4.6 

Inspection, measuring and test equipment 

# 

4.7 

Inspection and test status 

e 

4.8 

Control of nonconforming product 

e 

4.9 

Handling, storage, packaging and delivery 

• 

4.10 

Quality records 

e 

4.11 

Training 

e 

4.12 

Statistical techniques 
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ISO 9003 obviously applies to mass production of 
supplier-specified products. In addition to the deletion of 
subparagraphs in paragraph 4 that relate to process 
inspection, the remainder of ISO 9003 has been modified to 
reflect a less intrusive quality assurance program regarding 
purchasing and process control. Using ISO 9003 puts very 
little burden on the supplier other than final inspection and 
test requirements. 

C. DOD MILITARY QUALITY STANDARDS 

The primary standards cited in Department of Defense (DoD) 
contracts concerning quality assurance programs are Military 
Specifications MIL-Q-9858A (MIL-Q) and MIL-I-45208A (MIL-I). 
A brief overview of the guiding philosophy for each of these 
specifications is presented below. This overview is intended 
to provide background for comparison of these DoD 
specifications to the ISO 9000 standards; the comparison will 
be done in the next section of this study. 

1. MIL-Q-9858A: Quality Progreun Requirements. 

This specification is structured from the standpoint 
of the Government. It directs suppliers on what they shall 
consider when conforming to this specification, and it 
requires approval of the company's quality program by the 
Government. The following two paragraphs provide a 
perspective for this specification: 
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1.2 Contractual Intent. This specification requires the 
establishment of a quality program by the contractor to 
assure compliance with the requirements of the contract. 
The program and procedures used to implement this 
specification shall be developed by the contractor. The 
quality program, including procedures, processes and 
product shall be dociimented and shall be subject to review 
by the Government Representative. The quality program is 
subject to the disapproval of the Government 
Representative whenever the contractor's procedures do not 
accomplish their objectives. The Government, at its 
option, may furnish written notice of the acceptability of 
the contractor's quality program. 

1.3 Summary. An effective and economical quality program, 
planned and developed in consonance with the contractor's 
other administrative and technical programs, is required 
by this specification. Design of the program shall be 
based upon consideration of the technical and 
manufacturing aspects of production and related 
engineering design and materials. The program shall 
assure adequate quality throughout all areas of contract 
performance; for example, design, development, 
fabrication, processing, assembly, inspection, test, 
maintenance, packaging, shipping, storage and site 
installation. 


These two examples illustrate the somewhat negative 
writing style and all-encompassing nature of MIL-Q-9858A 
specification. There are "laundry lists" throughout the 
specification written to cover all possible eventualities that 
might affect the quality program of a company engaged in 
complex production processes. MIL-Q-9858A (which is 
reproduced on the left-hand side of Appendix B) offers a 
legalistic framework in much of its phrasing and words. A 
careful analysis of MII-Q-9858A leads one to conclude that the 
drafters of this specification were seeking to close as many 
legal challenges to their idea of a quality program as 
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possible. The goal of using the existing supplier quality 
assurance program to fulfill quality requirements of the 
GovernmenR becomes lost in all the Government oversight and 
inspections, required to conform to this specification. The 
end result is a requirement on the part of the Government to 
be constantly inspecting the quality assurance program to 
ensure compliance with all the various parts of this 
specification. Thus MIL-Q-9858A becomes an attempt to 
"inspect quality into the product" rather than build it in, 
using good quality processes that stress improving the 
existing production system. 

This specification becomes even more encumbering and 
intrusive when references within MIL-Q are reviewed, A 
partial list of these specifications, along with brief titles, 
is given below: 

• MIL-I-45208A -- Inspection system requirements 

• MIL-C-45662 -- Calibration system requirements 

• MIL-STD-105 -- Sampling procedures and tables for 

inspection by attributes 

• MIL-STD-109 -- Quality assurance terms and definitions 

This provides much more than broad guidance in how a 
supplier will establish his quality assurance organization. 
While the specification clearly says the contractor is 
responsible for this system, DoD is stipulating rules of 
operation throughout MIL-Q-9858A that limit a contractor's 
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options. The scope of this specification is revealed in Note 
8 . 1 : 

8.1 Intended Use. This specification will a^jply to 
complex supplies, components, equipments, and systems for 
which the requirements of MIL-I-45208 are inadequate to 
provide needed quality assurance. In such cases, total 
conformance to contract requirements cannot be obtained 
effectively and economically solely by controlling 
inspection and testing. Therefore, it is essential to 
control work operations and manufacturing processes as 
well as inspections and tests. 

This quote clearly shows that the intent of MIL-Q-9858A is to 
obtain total control over the process and operation of the 
supplier. There is an implication that if DoD inspects both 
the end product and the process a sufficient number of times, 
the quality of the product will eventually meet its 
expectations. 

In contrast to the idea of total control, DoD has 
promulgated MIL-I-45208A for those contracts that do not need 
to be as closely monitored because of the nature or complexity 
of the product. 

2. MIL>I-45208A: Inspection System Requirements. 

This specification contains fewer requirements than 
MIL-Q, but it is still based on inspection and detailed 
accounting of quality by the supplier. There is an attempt to 
cover as many contingencies as possible and to provide not 
only guidance, but decisions concerning possible scenarios 
that might arise during the performance of a contract. These 
decisions take the form of negative wording regarding any 
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deviations or foreseeable problems. An example is found in 
paragraph 2.2: 

2.2 Amendments and Revisions. Whenever this specification 
is amended or revised subsequent to its contractually 
effective date, the contractor may follow or authorize his 
subcontractors to follow the amended or revised document 
provided no increase in price or fee is required. The 
contractor shall not be required to follow the amended or 
revised document except as a change in contract. If the 
contractor elects to follow the amended or revised 
document, he shall notify the Contracting Officer in 
writing of this election. When the contractor elects to 
follow the provisions of an amendment or revision, he must 
follow them in full. 

This paragraph shows how the drafters of MIL-I-45208 
tried to incorporate all possible scenarios of a change to a 
product within the specification. The specification first 
states that a contractor may elect to follow a revision, 
provided there is no increase in price. Then it is 
hypothesized that a contractor might elect to follow only part 
of the revision and so that becomes forbidden. The hypothesis 
is taken one step further in assuming a contractor will fail 
to tell the Government what course of action has been 
implemented regarding amendments or revisions. To preclude 
this possible oversight on the part of a contractor, MIL-I- 
45208 requires that the Contracting Officer be notified in the 
event an amendment or revision is implemented. A careful 
reading of this paragraph shows how the original intent of the 
specification, which meant simply to state that a contractor 
does not have to follow revisions until they are formally 
added to a contract, has become an exercise in semantics. 
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This type of phrasing and writing pervades both MIL-Q-9858A 
and MIL-I-4520S. 

D. LITERATURE REVIEW 

In order for the researcher to understand the 
attraction of using international standards, and the movement 
within DoD to accept them, a survey of current literature 
regarding international quality standardization was conducted. 

1. Change In attitudes during 1980s. 

The early 1980s saw the realization that the 
predominate position of the United States as arms supplier to 
all its allies -- in particular, its European allies -- was 
being seriously challenged. The political climate dictated 
that jobs and technology development in Europe be protected by 
the European governments. This political drive to give equal 
status to the European arms industrial base led to serious 
concerns that soon the North Atlantic Treaty Organization 
(NATO) would be equipped with incompatible weapons across 
national boundaries. These concerns gave rise to the concept 
of Rationalization, Standardization and Interoperability 
(RSI) . The political climate of both Europe and the U.S. was 
intent on achieving maximum military effectiveness for the 
money expended. One way of achieving this effectiveness was 
to create joint efforts that reduced the duplication of 
research and development funds while increasing the potential 
mar]<et and numbers of weapons eventually deployed. At the 
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same time, the early 1980s saw the U.S. as the dominant 
producer of complex weapon systems for the free world. Along 
with this dominance came a certain amount of parochial 
interest on the part of American companies and DoD components 
dealing in the international arena. The U.S. Government and 
industrial base were viewed by our allies as using defense to 
protect and justify purely economic decisions. 

[Ref 6:p.7] 

This attitude, that the U.S. would establish contracts 
with NATO allies to maximize resource use and to ensure NATO 
compatibility in weapon systems, has been replaced. The new 
views on international contracting have to do with the 
realization that the United States is . dominant in many 
areas and thus must compete fci its foreign market share. 
This new attitude is summa''ized in the following citation: [Ref 
7:p.6] 


This year's NCM Week slogan (Contracting for a Competitive 
America) is not prophetic; it is descriptive of current 
real world events. The general downsizing of defense and 
high technology markets and the increasing competition 
from essentially equivalent foreign competitors are 
demonstrable indicators that the economic future offers 
rewards to those who succeed in international competition 
and penalties (in the form of decreased market share, 
fewer jobs, and less profit) to those who do not. 

International arms deals are now viewed as a 
competitive field. Industry and DoD now see foreign 
competitors as more of an equal and thus the U.S. can no 
longer dictate the terms of international arms sales. 
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2. Policy statements regarding quality assureuice in 

international contracts. 

The official DoD policy toward adoption of ISO 9000 
can be seen in several documents. First, at a broad policy 
level, there is DoD Instruction 5000.2 Part 6 Section Q (c) 
Participation in Standards Development Activities , which 
states; 

DoD Components will participate in standards development 
activities of non-Government standards bodies, both 
domestic and international, coordinating on such activity 
with other Federal Agencies. 

This policy, in the case of ISO 9000 standards, has 
been implemented through its inclusion in the Department of 
Defense Index of Specifications and Standards (DODISS) list of 
acceptable documents for use in contracts. 

The DoD 5000.2 is backed up with an Assistant 
Secretary of Defense memorandum dated 7 August 1989 that 
states: [Ref 8:p.l] 

I have been briefed on the series of quality assurance 
standards recently published by the International 
Organization for Standardization (ISO). After reviewing 
the benefits of adopting these ISO quality assurance 
standards, their wide acceptance around the world, and 
their implications in international trade, I have decided 
to follow an approach similar to that taken by NATO. I 
believe that this approach will provide maximum benefit to 
the Department of Defense (DoD) as well as American 
industry. 

More specifically, I want to adopt ISO standards, 9001, 
9002, 9003 in their entirety and develop supplemental 
military standards that will incorporate the appropriate 
ISO standards by reference and provide the requirements 
for a contractor quality program which may be lacking in 
the ISO standards. 
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The Army is the Service branch with cognizance over 
Quality Assurance for DoD, and thus has the lead in policy 
generation and modification for quality assurance. 

In a brief given by the Assistant Director, Office, 
Secretary of Defense International Quality Assurance, the 
following bullets are .listed on a viewgraph labeled "U.S. 
National Position on ISO QA Standards[Ref 9] 

• US has adopted ISO QA Standards: ANSI/ASQA Q-90 Series 

• US consensus is ISO Standards are insufficient 

• ISO Standards will be revised or reconfirmed in 1992 

• Task force to review Q-90 Series Standards established 

• Task force authorized under ANSI ASC A-1 committee 

• Task force to identify improvements in ISO standards 

• These improvements will be provided to US TAG [Technical 
Advisory Group] and to ISO TC176 [Technical Committee]. 

• Task force has met several times 

• DoD and defense industry are represented 

The U.S. Department of Commerce is also heavily 
involved in the promulgation of information regarding ISO 
Standards and in assessing the impact they will have when the 
European Community is formally created in 1992. The abstract 
shown below is from an article published by the Department of 
Commerce, National Institute of Standards and Technology, 
Standards Code and Information Program, Office of Standards 
Service: [Ref 10:p. ii] 
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This report provides information on the development, 
content and application of the ISO 9000 standards to 
readers who are unfamiliar with these aspects of the 
standards. It attempts to answer some of the most 
commonly asked questions on quality; quality systems; the 
content, application and revision of the ISO 9000 
standards; quality system approval/registration; European 
Community requirements for quality system 
approval/registration; and sources for additional help. 

The literature cited above is meant to give a general 

sense of what the U.S. Government and in particular the DoD 

position is with regards to ISO 9000. It is certainly not an 

exhaustive survey of all the policy issues and actions taken 

or proposed, but it does represent the largest force within 

this area of Quality Assurance. 

3. Concerns regarding the use of ISO 9000 and the manner 
in which ISO 9000 Standards are changed. 

During the literature review, documents that outline 
the concerns and perceived deficiencies in ISO 9000 standards 
were located. These documents are best summarized by a 
concise overview used during a briefing entitled "ISO 9000 
Quality Assurance standards" given during a training session 
for Quality Assurance Representatives (QARs) in the Defense 
Contract Management District (DCMD) Northeast. The overview 
was titled "ISO QA Standards Concerns" and listed the 
following bullets:[Ref ll:p.l2] 


• Contractor certification 

• Requirements not adequately covered by ISO/AI'JEI Standards 

• DoD guidance document [to be issued in near future] 


22 





• Training [QAR training in philosophy and application of 
ISO Standards] 

The issues regarding deficiencies in ISO 9000 
Standards are the purview of the International Standards 
Organization Technical Coininittee 176 (TC176) . TC176 is 

responsible for overseeing the modifications and additions 
made to ISO 9000 Standards. The four Strategic Goals of the 
TC176 were recently defined by using "test" statements. The 
goals and associated "tests" are shown below.[Ref 12:p.33] 
Goal: Universal Acceptance 

Tests for the goal of universal acceptance: 

• The standards are widely adopted and used, worldwide. 

• There are few complaints from users in proportion to the 
volume of use. 

• Few sector-specific supplementary or derivative standards 
are being used, or developed. 

Goal: Current Compatibility 

Tests for current compatibility: 

• " ^rt Number" supplements to existing standards do not 
change or conflict with requirements in the existing 
parent document. 

• The numbering and clause structure of a supplement 
facilitate combined use of the parent document and the 
supplement. 

• Supplements are not stand-alone documents, but a?re to be 
used with their parent doctunent. 


Goal: Forward Compatibility 




Tests for forward compatibility: 


• Revisions affecting requirements in existing standards are 
few in number and minor or narrow in scope, 

• Revisions are accepted for existing as well as new 
contracts. 


Goal: Forward Flexibility 
Tests for forward flexibility: 


• Supplements are few in number, but can be combined as 
needed to meet the needs of virtually any 
industry/economic sector or generic category of products. 

• Supplement or addendum architecture allows new features or 
requirements to be consolidated into the parent document 
at a substantial revision, if the supplement's provisions 
are found to be used (almost) universally. 


The two documents cited above illustrate the concern 
many ISO members have regarding modifications or changes to 
ISO 9000. The standards must be applicable across national 
boundaries and they must cover all types of industry. ISO 
developed a method for dealing with modifications of ISO 9000 
in order to accommodate the resolution of some of these 
concerns. In the case of DoD, the stated policy is to issue 
and impose additional requirements, with regard to the 
contractor's quality system as needed, to ensure a contractor 
has an acceptable quality system in place. 
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B. SUMMARY 


The structure and philosophy of ISO 9000 as explained 
above is designed to facilitate continuous improvement in 
quality. Thus it is a step closer to a Total Quality 
Management system as well. On the other hand, the existing 
Military Standards tend to inspect quality into the product, 
as opposed to concentration on improvement of the process. 

The literature search revealed a concerted policy within 
DoD to encourage and educate both contractors and DoD 
personnel in the application of ISO 9000. At the same time, 
DoD is reserving the right to modify ISO 9000 programs by 
imposing existing or new quality standards as appropriate for 
a given contract. 

The following chapter compares MIL-Q-9858A to ISO 9001 and 
MIL-I-45208 to ISO 9002. This comparison highlights 
differences between the two sets of standards, in order to 
gain a better understanding of the changes that will result 
from imposing ISO 9000 in DoD contracts. 
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III. COMPARISON 


A. INTRODUCTION 

The structure of the ISO 9000 series of standards makes it 
easy to compare ISO 9001 to MIL-Q-9858A. But ISO 9002 does 
not compare well to MIL-I-45208, while ISO 9003 can be thought 
of as equivalent to inspection by the contractor of his own 
product or, more broadly, as commercial end-item inspection. 
This researcher prepared a comparison of ISO 9001 to MIL-Q- 
9858A and a comparison of ISO 9002 to MIL-I-45208. These 
comparisons are presented in Appendices B and C of this 
thesis. 

Appendix B compares individual paragraphs of ISO 9001 to 
corresponding paragraphs of MIL-Q-9858A, while Appendix C 
compares ISO 9002 to MIL-I-45208 on a paragraph-by-paragraph 
basis. Thus, a detailed comparison of the standards in 
regards to a particular subject matter is available by careful 
scrutiny of Appendices B and C. After completion of these 
comparisons, the researcher discovered a similar collation 
completed by the Office of the Assistant Secretary of Defense 
(Production and Logistics). In reviewing the researcher's 
comparison as opposed to that of the OSD(P&L), it is evident 
that the two documents agree only in principle as to possible 
alignments of paragraphs between ISO 9001 and MIL-Q-9858A. 
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This wide disparity illustrates the subjective nature of both 
documents. It is possible to interpret a given paragraph in 
ISO or MIL-STD in several different ways. The OSD(P&L) 
comparison is discussed in detail below, and Appendix D of 
this thesis is a matrix of the OSD(P&L) alignment of 
paragraphs between ISO 9001 and MIL-Q-9858A. 

It should be noted that Appendices B and C adopted the ISO 
9000 organization with the right-hand side of the table 
reflecting the ISO 9000 structure. MIL-Q-9858A (in Appendix 
B) and MIL-I-45208 (in Appendix C) are then applied to the ISO 
Standards on a best-of-fit type comparison on the left side of 
the appendix. 

B. GENERAL OBSERVATIONS 

After reviewing the detailed comparison of ISO to MIL 
STD, three general observations were made by this researcher: 

1. The ISO standards use the phrase "as appropriate" 
while the U.S. specifications attempt to list all possible 
situations to which they might wish to apply a given 
paragraph. 

2. The ISO standards use a clear outline format, 
which separates the main subjects into short easy-to-read 
statements, while U.S. specifications employ long narrative 
paragraphs. 

3. The ISO standards take an overall approach and 
provide broad management structure to the programs they are 
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seeking to encourage. U.S. specifications appear to direct 
contractors to take specific action in areas that have caused 
problems in the past. 

A second view of the general differences between ISO 9000 
and Military Standards was given during a panel discussion at 
the 23rd Annual Industry/Government Quality Liaison Meeting 
held at Dan” rs, Massachusetts. The following concise 
comparison of ISO 9000 to MIL-Q strengths was presented as a 
panel discussion subject.[Ref 13:p.6] 

ISO 9000 and MIL-0-9858A are generally equivalent 
ISO 9000 is stronger in; 

1. Design control 

2. Management commitment 

3. Personnel training/qualification 

4. Internal quality audits 

MIL-0-9858 is Stronger in; 

1. Calibration and measurement 

2. Control of nonconforming supplies 

3. Government review of QA program 

4. Cost of quality 

Both are weak in continuous process improvement. 

C. RESEARCHER'S COMPARISON OF ISO 9001 TO MIL-Q-9858A 

ISO 9001 compares favorably with MIL-Q-9858A. This 
researcher does not detect any significant differences between 
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the two. Both documents cover the same areas of process and 
design control. 

ISO 9001 does not currently contain the "layering effect" 
that is inherent in MIL-Q-9858A. This layering effect is 
where MIL-Q-9858A refers to other specifications or handbooks 
that amplify and add more detail to MIL-Q-9858A. There is 
resistance to additional amplification within the ISO 
organization and membership. As discussed above, the ISO 
TC176 has established a set of strategic goals that are 
designed to avoid the micro-management of MIL-Q implementation 
practices. One of the goals of ISO is to remain at a basic 
level that can -y to many different industries and 
situations. 

D. RESEARCHER'S COMPARISON OF ISO 9002 TO MIL-I-45208 

ISO 9002 is far more comprehensive and provides more 
direction than does MIL-I-45208 and is not a close replacement 
for it. ISO 9002 actually exceeds the MIL-I-45208 
requirements in all areas, and is virtually identical to ISO 
9001 (with the removal of the paragraphs in ISO 9001 
concerning design quality control). 

All other provisions regarding process control and 
documentation remain the same between ISO 9001 and ISO 9002. 
The use of ISO 9002 in place of MIL-I-45208 has the potential 
to increase expenditure of resources by contractors to comply 




with it, over and above the costs of complying with MIL-I- 
45208. 

E. RESEARCHER'S ASSESSMENT OF SECTIONS IN MIL-Q-9858 AND MIL- 

1-45208 WITH NO CORRESPONDING ISO SECTIONS 

A review of the text for those paragraphs that have no 
corresponding ISO wording, reveals that these paragraphs do 
not deal with defining a quality program. Rather, they are 
directives designed to provide the Government additional legal 
rights to conduct a detailed or more intrusive oversight of 
subcontractors. They also ensure legal right to the use of 
contractor equipment in conducting in-process inspections. 
These paragraphs are not deemed by this researcher to be 
applicable to the overall effectiveness of a quality program. 

1. MIL-Q-9858A. 

The paragraphs of MIL-Q-9858A that do not have 
corresponding ISO 9001 paragraphs are listed, along with their 
titles, below: 

• 3.6 Cost Related to Quality 

• 4.4 Use of Contractor's Inspection Equipment 

• 4.5 Advanced Metrology Requirements 

• 7.7 Government Inspection at Subcontractor Facilities 

The complete text of these paragraphs can be found at 
the end of Appendix B. 
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2. MIL-I--45208. 


The paragraphs of MIL-l-45208 that do not have 
corresponding ISO 9002 paragraphs are listed, along with their 
titles, below: 

• 3.8 Qualified Products 

• 3.11 Government Inspection at Subcontractor or 

Vendor Facilities 

• 3.11.1 Government Inspection Requirements 

• 3.13 Government Evaluation 

The complete text of these paragraphs can be found at 
the end of Appendix C. 

F. OSD(P&L) COMPARISON OF ISO 9001 TO MIL-Q-9858A 

The OSD(PScL) conducted a comparison, resulting in 38 
comments on the differences between ISO 9001 and MIL-Q-9858A. 
The comments have been broken down by this researcher into 
broad areas of the standards to which they apply, and are 
presented below. This OSDIP&L) comparison is meant only as a 
reference document and not as a policy statement. 

1. Scope. 

This is paragraph 1.0 in ISO 9001 and paragraph 1 in 
MIL-Q. The paragraphs are intended in both standards to state 
the applicability of the standard. OSD{P&:L) comments are: 
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• Mil-Q-9858 states that the contractor is responsible for 
compliance with all provisions of the contract and for 
furnishing specified supplies and services which meet all 
the requirements of the contract. ISO does not have this 
statement. 

• ISO provides for tailoring of specific contractual 
situations. MIL-Q-9858 does not have such a provision. 

• ISO states that it is primarily aimed at preventing 
nonconformity. MIL-Q-9858 does not state this primary 
aim. 

• ISO applies when the contract specifically requires design 
effort and the product requirements are stated principally 
in performance terms or they need to be established. MIL- 
Q-9858 applies to all supplies or services when referenced 
in the contract. 

• MIL-Q-9858 requires the quality program, including 
procedures, processes and product, to be subject to review 
by the Government Representative, to have access to and 
the right to review and evaluate all aspects of the 
contractor's quality program. ISO does contain provisions 
for purchaser review and evaluation except where agreed to 
contractually, (paragraph 4.16) 


The OSD(P&L) comment concerning the prevention of 
nonconformity as a goal for ISO, while it is not for MIL-Q, 
addresses the philosophical difference between the two 
standards. ISO is concentrating on the development of an 
organization that is set on improvement and the prevention of 
nonconforming material; MIL-Q concentrates only on inspections 
and detection of nonconforming material. The last comment 
regarding access to evaluate the quality assurance system of 
a company is really a legal issue that has been buried in the 
MIL-STD, instead of being addressed in contractual language 
where it appropriately belongs. 
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2. Quality Systems Requirements/Quality Policy. 

This is a paragraph in ISO 9001 to which MIL-Q-9858 
has no equivalent. The OSD(P&L) comments are: 

« ISO requires the supplier's management to define and 
document its commitment to quality. MIL-Q-9858 does not 
require this. 

This provision of ISO 9001 clearly delineates the 
heart of the difference between it and MIL-Q-9858. ISO is 
designed to create an organization that is committed to 
continuous improvement of quality, and that commitment has to 
come from the top echelons of management. MIL-Q-9858 is an 
attempt to put quality into the process/product through 
continuous inspection, 

3. Organization/Management. 

The organization and responsibility within the Quality 
Assurance department of a company is addressed in both 
standards. OSD(P&:L) comments are: 


• ISO requires the responsibility, authority, and the 
interrelation of all personnel who manage, perform, and 
verify work affecting quality be defined. MIL-Q-9858 
requirp's that the responsibility and authority of 
personnel performing quality functions be defined, 
(paragraph 3.1) 

• ISO requires the supplier to assign trained personnel for 
verification activities. MIL-Q-9858 does not specifically 
require the assignment of trained personnel. 

• ISO requires that verification, including inspection, 
test, and monitoring, be carried out by personnel, 
independent of those having direct responsibility for the 
work being performed. MIL-Q-9858 requires that personnel 
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performing quality functions shall have the organizational 
freedom to identify and evaluate quality problems, 
(paragraph 3.1) 

• ISO requires that a management representative be 
responsible for ensuring that the requirements of the 
quality system are implemented and maintained. MIL-Q-9858 
indicates that the fulfillment of the quality program 
requirements is not the responsibility of any single 
contractor's organization, function or person. 


The ISO standards are written to improve the quality 
assurance system within a company. An important element of 
this program is the initiation of a program of continuous 
improvement. The concept of continuous improvement requires 
that people be trained and qualified to operate and maintain 
the quality system described by ISO 9000 standards. The MIL-Q 
is less stringent concerning employee qualification, because 
it has an inherent attitude that more inspection will reveal 
the defects and thus overcome the failures that may exist in 
the level of training within the company's quality assurance 
system. 

4. Quality System/Program. 

These sections of the standards deal with the 
requirements for establisning a quality assurance system 
within the company. ISO requires the establishment of a 
system that covers all phases of the production process, while 
MIL-Q-9858 is written in terms of a program that is an end in 
and of itself. OSD(PScL) comments are: 
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• ISO provides that consideration needs to be given for the 
preparation of a quality plan (defined in ISO 8402 as a 
document setting out the specific quality practices, 
resources and sequences of activities relevant to a 
particular product, service, contract or project). MIL-Q- 
9858 does not have any provision for a quality plan. 

• ISO provides that consideration needs to be given for the 
compatibility of the design, the production process, 
installation, and the applicable documentation. MIL-Q- 
9858 does not provide for the compatibility of the design 
and production process. 


5. Contract Review/Design, Control/Product, 

Identification and Traceability. 

These are three topics covered by ISO and not covered 
in MIL-Q-9858. The OSD(P&L) comments are: 


• ISO requires the supplier to review each contract and 
maintain records of these reviews. MIL-Q-9858 requires 
the contractor to conduct a complete review of the 
requirements of the contract. Records of these reviews 
are not specifically required. 

’ ISO devotes an entire section to design control. MIL-Q- 
9858 does not address design control. 

• ISO requires the identification of the product from 
applicable drawings, specifications, or other documents 
during all stages of production, delivery, and 
installation. Such identification is not required in MIL- 
Q-9858. 


In both these cases, ISO is covering the entire 
process that a contractor must complete to achieve an 
acceptable end product. It attempts to avoid an unacceptable 
end product by ensuring that the entire production process is 
under control. The MIL-Q relies on an inspection program to 
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detect and ultimately remove defective products after they 
have been produced. 

6. Document Control/Drawings, Documentation and Changes. 

The procedures prescribed in the standards to control 
changes to the technical data package are fundamentally 
different. MIL-Q advocates a broader approach in evaluating 
the adequacy of records keeping with regard to engineering 
changes, while ISO has specific steps to be followed in this 
area. The OSD(P&L) comments reflect this difference: 


• MIL-Q-9858 requires a procedure for the evaluation of the 
adequacy of design drawings and specifications in relation 
to standard engineering and design practices, and with 
respect to the design and purpose of the product. This 
requirement is not in ISO. 

• ISO requires that a master list or equivalent document 
control be established. ISO also requires that documents 
be re-issued after a practical number of changes have been 
made. These requirements are not in MIL-Q-9858. 

• MIL-Q-9858 extends the contractor's responsibility to 
drawings and changes provided by subcontractors and 
vendors. This requirement is not in ISO. 


The ISO 9001 standard gives more detailed guidance in 
establishing control over the technical data package. The 
MIL-Q-9858 leaves this area up to the contractor's established 
systems. It tries to use an industry best-practices approach 
to document control, based on the theory that there is such a 
wide discretionary range in approaching this problem that 
whatever system the contractor is using is acceptable, as long 
as it is close to industry practices. 
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7. Purchasing Control. 

OSD{P&L) coiranents: 


• ISO requires the establishment and maintenance of records 
of acceptable subcontractors. Such records are not 
required in MIL-Q-9858. 

• MIL-Q-9858 provides for inspection by the Government at 
the subcontractor's plant. ISO provides for the purchaser 
or his representative to verify at source, when specified 
in the contract, that the purchased product conforms to 
specified requirements. (para 4.6.4) 

• MIL-Q-9858 requires the use of test reports, inspection 
records, certificates and other suitable evidence relating 
to the subcontractor's control of quality. MIL-Q-9858 
requires the contractor to have procedures for the early 
information feedback and correction of subcontractor 
nonconformances. This is not required in ISO. 

• MIL-Q-9858 includes requirements for chemical and physical 
testing and recording in connection with the purchase of 
raw materials by suppliers. Such testing is not required 
in ISO, 


The ISO standards reflect an attitude that contractors 
should use a "best value" approach in selecting their 
subcontractors. The definition of "best value" is the 
attainment of maximum benefit from money spent in terms of 
satisfying requirements of the contract. Thus, the 

establishment of long-term relationships between contractor 
and supplier is not looked upon in Europe as being "collusive" 
as it frequently is in the U.S. DoD stresses use of the 
lowest bidder for subcontract work, creating a barrier to 
development of long-term relationships. If a subcontractor is 
not the lowest bidder for future business, the prime 
contractor is obliged to select the lowest cost proposal. 
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regardless of past performance considerations. The value of 
a proven subcontractor carries little weight when prime 
contractors are evaluated for control of subcontracts by DoD. 
The MIL-Q-9858A is relying on incoming inspection to detect 
defective products, rather than trying to establish a rapport 
with a subcontractor to eliminate substandard material. 

8. Purchaser Supplied Product/Government Property. 

ISO 9001 has a brief three-sentence paragraph that 
requires a procedure be set for receipt, storage and reporting 
of problems 

regarding purchaser supplied material. There are no specific 
directions as to protection and disposition of such material. 
Meanwhile, MIL-Q-9858A has three lengthy paragraphs that 
explain what action will be taken if problems arise with 
Government Furnished Property (GFP). MIL-Q-9858A attempts to 

cover all contingencies of this situation. The OSD(P&L) 
comment is: 

• MIL-Q-9858 requires the contractor to provide more 
specific protection of Government-furnished material. The 
control of purchaser supplied product in ISO is more 
general. 

9. Process Control/Work Instructions. 

These sections of the standards discuss how a company 
documents the division of labor within the manufacturing 
process. In essence it is how the bill of material and labor 
sheets are written. OSD(PStL) comments; 
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• ISO requires work instructions defining the manner of 
production and installation only. MIL-Q-9858 requires 
work instructions for purchasing, handling, machining, 
assembling, fabricating, processing, inspecting, testing, 
modification, installation, and many other treatments of 
product, facilities, standards, or equipment. 


The OSD{PScL) comment on this section of the standards 
would lead one to conclude that it covers less than MIL-Q- 
9858. In reality, ISO 9001 encompasses more than MIL-Q-9858 
in this area because it focuses on covering the entire 
manufacturing process with controlled written procedures. The 
MIL-Q-9858 states the requirements for this area in terms of 
"all work affecting quality." Since ISO covers the entire 
process, i.e., all work affecting quality, both standards are 
identical in intent. 

10. Inspection and Testing/Manufacturing Control. 

ISO 9001 concentrates on establishing a process that 
assures an acceptable incoming product from suppliers. MIL-Q- 
9858 relies on inspection to single out nonconforming 
material. OSD(P&L) comments: 

• MIL-Q-9858 requires raw materials to conform to the 
applicable physical, chemical, and other technical 
requirements. Laboratory testing shall be employed as 
necessary. ISO does not require such testing. 


11. Final Inspection and Test. 

The OSD(P&L) comment in this area is: 


• MIL-Q-9858 requires reporting to designers any unusual 
difficulties, deficiencies or questionable conditions 
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during final inspection and testing. ISO does not require 
this feedback. 


The OSD comment in this area fails to view ISO 9001 in 
its entirety. ISO requires more formalization of 

feedback/corrective action procedures than does MIL-Q-9858. 
It simply does not list those corrective action steps at this 
point in the standard. The philosophy that supports ISO 9001 
would say that nonconformance at final inspection means there 
was a failure of the quality assurance 

system because it failed to prevent the nonconformance in the 
first place. 

12. Measuring and Test Equipment. 

The OSD(P&L) comments: 

• MIL-Q-9858 is more specific and elaborate in the area of 
measuring and test equipment by requiring conformity with 
MIL-C-45662. 

• MIL-Q-9858 requires the contractor to make his gauges, 
measuring and testing devices available to the Government. 
ISO does not have this requirement. 

The first comment listed above seems to be a moot 
point. If a company is using ISO 9001, it can supplement it 
with other standards. For example, a contract written using 
ISO 9001 as a requirement, still can designate MIL-C-45662 
(calibration) as a required technical specification. 
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The second comment listed above concerning Government 
access is another example of where MIL-Q-9858 was used as a 
vehicle for contractual requirements that belong elsewhere in 
a contract, not in the quality assurance standard. 


13. Control of Nonconforming Material. 

In the area of the nonconforming material, ISO 9001 
and MIL-Q-9858A take different approaches to resolving rework, 
due to the difference in the philosophy of the standards. ISO 
views rework as a source of information back to the process 
that improvement is possible. Thus ISO specifies that rework 
effort is to undergo the same inspection as the original 
effort. MIL-Q-9858A calls for separate procedures to resolve 
rework, thus removing a feedback loop back to the original 
production process. The OSD(P&L) comments are: 


• MIL-Q-9858 states that the acceptance of nonconforming 
supplies is a prerogative of, and shall be as prescribed 
by, the Government. ISO does not have an equivalent 
statement. MIL-Q-9858 requires that repair or rework be in 
accordance with documented procedures acceptable to the 
Government. Procedures for repair or rework are not 
required by ISO. ISO requires that repaired and reworked 
product be re-inspected in accordance with documented 
procedures. MIL-Q-9858 does not require re-inspection. 

• MIL-Q-9858 requires data associated with the costs and 
losses, in connection with scrap and rework, be made known 
to the Government. This is not required in ISO. 
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14. Hemdllng, Storage, Packaging and Delivery. 

The intent of both standards is the same in this area; 
the comment below seems to this researcher to be one of 
semantics: 


• MIL-Q-9858 requires periodic inspection for the prevention 
of deterioration or damage in storage. ISO requires 
periodic assessment of product in stock to detect (not 
prevent) deterioration. 


15. Quality Records. 

Because of the emphasis in MIL-Q-9858 on final product 
inspection, the MIL-STD sets specific requirements for 
records- keeping regarding final inspection. ISO relies on 
feedback and continual improvement, leaving open the exact 
form of the data, so long as it is brought to the attention of 
personnel responsible for the area being tested. The OSD(P&L) 
comments are: 


• MIL-Q-9858 requires records to indicate the nature of the 
observations, together with the number of observations 
made and the number and type of deficiencies found. ISO 
does not specify the contents of the records. 

• MIL-Q-9858 requires records to be used as a basis for 
management action. ISO does not have this requirement. 

• MIL-Q-9858 does not have a requirement for internal 
quality audits as required in ISO. 


16. Training and Servicing. 

These are two areas not covered by MIL-Q-9858. 
OSD(P&L) comments: 
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• ISO requires a procedure for the ider)^ if ication and 
provision of training for all personnel performing 
activities affecting quality. MIL-Q-9858 does not have 
this requirement. 

• ISO requires procedures to assure that servicing 
requirements are met. MIL-Q-9858 does not cover 
servicing. 


17. Statistical Techniques. 

The ISO Standards require statistical methods of 
control be applied to all processes covered by the standards. 
MIL-Q, on the other hand, uses sampling to conduct 
inspections. Again it is the fundamental atticude of either 
controlling the process [ISO] or inspecting the end product 
and removing unacceptable material [MIL-Q]. 0SD(P6cL) 
comments are: 


• ISO requires the use of statistical techniques for process 
capability and for product characteristics. MIL-Q-9858 
does not have provision for such statistical techniques. 
MIL-Q-9858 allows the contractor to use sampling 
procedures for product acceptance. ISO does not 
specifically allow sampling. 


18. Costs Related to Quality. 

This is the one major area found in MIL-Q-9858 which is 
totally absent in ISO. OSD(P&L) comments: 


• MIL-Q-9858 requires the contractor to maintain and use 
quality cost data. ISO does not require quality cost 
data. 
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G. SUMMARY 


An armchair comparison of ISO 9000 to Military Standards 
leaves this researcher with the impression that a change to 
ISO would not be difficult, and the resulting Quality 
Assurance program changes would aid in improving quality and 
reducing costs. A survey of contractors, described in the 
next chapter, reinforces this impression. 
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ZV. DATA ANALYSIS/INTERPRETATION 


A. INTRODUCTION 

The data presented in this study were gathered through a 
sampling of 24 managers or directors of quality assurance 
departments from companies that are ISO 9000 registered and 
were selected on the likelihood that they were familiar with 
MIL-STDS. Each survey was conducted by telephone and lasted 
from 20 to 45 minutes. The respondents were encouraged to 
answer freely on a nonattribution basis. The aggregate 
listing of respondents is presented in Appendix A. 

In order to reach companies primarily concerned with DoD 
contracting, only those companies listed in the following 
categories were contacted: [Ref 14] 

• Rubber and Miscellaneous Plastics Products 

• Primary Metal Industries 

• Fabricated Metal Products 

• Industrial and Commercial Machinery and Computer Equipment 

• Electronic and Other Electrical Equipment 

• Measuring, Analyzing and Controlling Instruments 

• Wholesale Trade - Durable Goods 

The researcher was interested only in contacting those 
companies actively involved in DoD contracts and that were 
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aware of ISO 9000 Standards. As a result, the data are 
heavily biased toward positive coiranents regarding I~r aOOO 
because the firms contacted have already demonstrated a strong 
commitment to ISO 9000 by completing the registration process. 

Of the 25 companies interviewed, 20 of them are registered 
to the ISO 9001 standard and five are registered to the ISO 
9002 standard. All of the respondents were managers or 
directors of quality assurance departments within their 
companies and all had recent experience with the registration 
process. In many cases the person interviewed was responsible 
for planning and achieving ISO registration, and was thus very- 
familiar with the problems faced by a company attempting to 
achieve registration. 

B. THE RESPONSES 

1. Question One. 

Does your cos^any currently, or has it in the past, 
performed DoD contracts that specified con^liemce with MIL-Q- 
9858A or MIL-I-45208? 

a. Discussion 

The purpose of this question was to determine if 
the person interviewed was familiar with MIL-Q-9858 or MIL-I- 
45208 and if the company fell into the target group of DoD 
contractors. 
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Jb. Analysia 


In 24 of the 25 companies interviewed, the person 
contacted was familiar with MIL-Q-9858. This familiarity 
included compliance with MIL-STDS on current or previous 
contracts and a thorough knowledge of the quality assurance 
system their company designed in order to comply with MIL-Q- 
9858 or MIL-I-45208. Of this group, only 16 were actively 
involved in DoD contracts at the time the interview was 
conducted. 

2. Question Two. 

Was your company aware of DoD's policy to use ISO 9000 
in contracts, and if so, was it a factor in your company's 
decision to attain ISO registration? 
a. DiacuBsion 

This question was asked in order to get a feel for 
the level of awareness in industry with regard to ISO 
implementation by DoD. There were eight people interviewed 
who knew that DoD was reviewing the use of ISO 9000 in DoD 
contracts, and only three who knew that DoD was contemplating 
the replacement of MIL-Q-9858 with ISO 9001. The official 
position, as expressed by the Assistant Secretary of Defense 
in 1989, was completely new to all 25 respondents. When asked 
if DoD policy on ISO 9000 was a factor in getting registered, 
only one person said it was, and he quickly stated that it was 
only a minor factor. 
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b. Analysis 

The survey made it clear that the driving force 
for a firm to become registered is centered on remaining 
competitive in the global marketplace, in order to sell to the 
European Community after 1992. None of the comments indicated 
that consideration of future DoD business was a factor in the 
decisions. The decision to become ISO 9000 registered is 
obviously based on a desire for additional international 
sales. Some of the comments regarding this international 
marketplace are paraphrased below; 


• We went after 9001, [because] we are a U.S. company that 
does a lot of work in Europe and [we] export a lot to 
Europe. There is a theoretical requirement that by 
December of '92, ISO 9000 will be required to do business 
within the European community. So ISO 9000 is driven by 
the commercial customers in Europe. 

• [ISO 9000] is an advantage in the international [markets]. 

But it [ISO 9000] will become it [quality assurance] in 
the U.S. in very short order, I can see it. Our 

competition is scrambling right now because we have it and 
they don't. And we are going to just start flaunting it 
from a marketing point of view. 

• I think the primary impetus of this activity is that we 

have a high percentage of international (especially 
European) sales. And it is part of the Europe '92 

initiative. We felt this was important. 

• A number of years ago we started to pick up on this kind 

of requirement in the international community. We deal 
with a lot of major contractors -- international 
contractors -- and we started to see a lot more stringent 
quality requirements come through with regard to 
contracts. So we looked to see what was out there, and 

found that this standard had been revised in '87 and that 

a number of countries (40-some-odd of them) had also 
pretty much duplicated their national specs, such as our 
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ANSI ASQ C90, so at any rate, that is why we started 
considering it and that's why we did it. 

• ... by virtue of us trying to maintain the ability to sell 
in Europe. So the influence from the European Economic 
Community had influence as well. 

• ... ISO standard has been applicable since '87 and [the 
motive] was more that; if you intended doing business in 
the European Economic Community, you should address that 
criterion. 

• We became ISO 9000 qualified for a specific commercial 
sale. A customer of ours was looking for a marine 
industrial engine. It was a European customer and 
required that we be ISO 9000 certified. 


In addition to the marketing drive to remain 
competitive, the researcher also discovered that, for some 
companies, ISO 9000 is simply their next step down a road to 
continuous improvement and ultimately Total Quality 
Management. This situation is illustrated by the following 
paraphrased comments: 


• ... we were in the throes of, and still are in the throes 

of, finalizing implementation of a total quality 
management system. Well, we have about 10 different 
subsections in our quality management program. And there 
are 63 different clauses that we try to follow and about 
80 percent are required by ISO. So getting the ISO 

registration brought us to better than 80 percent into 
implementing our total quality management system. It gave 
us a nice independent third-party evaluation of where we 
were in implementing our own quality program. So that 
fortified our whole quality organization, made us feel 
pretty good. We were further than we realized. 

• We originally started with what they call QMI -- it's a 
division of Canadian Standards Association. It's like the 
ASME with Canada. Now they're going to all QMI or now ISO 
9000, but some three years ago we went with the QMI and 
passed this Canadian audit, and it was just a very minute 
revising of our manual to get into the ISO 9000. 









It should be remembered that this group of 
companies is in a select minority. There are only 225 U.S. 
firms registered to the ISO 9000 Standards and they are the 
very early leaders in a growing movement toward international 
standardization of quality. The firms contacted were selected 
on the likelihood that they were involved in high technology 
fields of production. The ISO 9000 Standards are more easily 
implemented in the manufacturing environment, and return the 
most benefit to firms engaged in high technology fields 
requiring a high degree of quality control in their production 
processes. 

3. Question Three. 

If DoD puts ISO 9000 requirements in a contract, would 
your company also agree to use your ISO registrar as an 
arbitrator of disputes between your firm and DoD? 
a. Discussion 

This question was designed to support or refute a 
recommendation that will be discussed in the next chapter of 
this thesis. It was explained during the interview that the 
ISO registrar would be contacted only as a last resort in the 
event of a deadlocked position arising between the Government 
and contractor. In addition, the loser would pay the expense 
of having retained the arbitrator. This payment provision was 
added to ease concerns that the Government would stonewall a 
contractor and drag all quality disputes before a registrar, 
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leaving the bill for registrar services to the contractor for 
payment. This entire suggestion of incorporating the ISO 
registrar as a last-resort arbitrator was intended to provide 
a new method of handling quality disputes with contractors, in 
such a manner as to reduce cost and delay factors associated 
with some of the really difficult disputes between DoD and 
contractors over quality issues. 
b. Analyaia 

All 25 respondents to this survey were 
enthusiastic supporters of implementing ISO 9000 Standards in 
DoD contracting, and for using registration as certification 
that a contractor's quality system was satisfactory for DoD 
contract purposes. Thus pre-award survey audits, to determine 
if a contractor has an acceptable quality system in place, are 
no longer performed. If a contractor has current ISO 9000 
registration, his quality system is automatically acceptable 
on a pre-award survey. 

The issue of arbitration was suggested by the 
researcher as a means to achieve even more benefit from a 
contractor being ISO 9000 registered. Arbitration becomes 
applicable in those situations where a Government Quality 
Assurance Representative (QAR) believes the contractor is not 
adhering to the quality policies and system used during the 
registration process. A complaint filed by a QAR with a 
company's registrar would jeopardize the company's ISO 9000 


51 




registration, and any commercial business requiring that 
registration. The final tally on the arbitration issue showed 
14 in favor of it, eight opposed, and two interviewees who 
refused to commit either way. The main reason for opposing 
the arbitration appeared to be that it was unnecessary because 
disputes should be settled before they reach arbitration. A 
sample of comments along these lines is paraphrased below: 


• I don't think it should be necessary to bring a third 
party into the picture. It should be something that we as 
the company and the Government should be able to resolve 
between us. 

• Any company, I think, would not want to have an outside 
party control their destiny. If there is a disagreement, 
they want to have the ability to negotiate it without 
having someone else come in and say you're right or wrong. 


• I really rather see it worked out between the supplier and 
the customer and come to a consensus, rather than bring in 
a third party. 

• ... if you accept the thesis that anybody that comes in 
should be thoroughly knowledgeable and professional in the 
auditing art or science, or both, then disputes that must 
be taken to arbitration should be almost nonexistent. I 
am a certified quality auditor, and I feel I should be 
able to resolve issues like that with a qualified auditor 
who comes in my building. I know his business as well as 
he does and we ought to be able to resolve those issues; 
if not, then I feel comfortable that I wouldn't take 
something to arbitration unless I really felt I was on 
secure ground. 

• ... basically, nobody likes to air his dirty laundry 
where it might get back to the customer. It could get to 
the competition somehow, somebody, might just pass it 
along. I always worry about that sort of thing getting 
into the so-called wrong hands, whatever that means. 

• We don't want disputes to reach that level. You are 
really asking for an independent mediator. 
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• I would think that before [arbitration] I would prefer to 
escalate it internally. Have more senior quality officers 
be involved, rather than going to external arbitration. 

• Well, by and large, I think the military side and the 
commercial side, the third party side, of auditing are 
pretty much in agreement on these things. After 20 years, 
I haven't seen things that couldn't be thought through and 
worked out at the plant level. 

• I think the first move should not be going to UL. The 
first move should be coming to the contractor, giving the 
contractor an opportunity to correct the oversight. If 
there is dispute on interpretation of what the program is 
requiring, that should be resolved before anything else 
occurs. 


The predominate response to the question of using 
arbitration was that it should be unnecessary. Yet, when 
asked, many of the people interviewed agreed that arbitration 
as a last resort was a good idea. The Contracting Officer who 
puts ISO 9000 into a contract may want to consider the use of 
ISO 9000 registrars as arbitrators. The idea of arbitration 
is particularly appealing when dealing with contractors or 
industries that have a history of sending quality problems up 
to higher levels of management for resolution. The data 
indicate that use of registrars as arbitrators would not 
create a prohibitive barrier for ISO 9000 registered 
contractors bidding on a contract. 

In addition, the ISO 9000 philosophy, which says 
that customers should receive the quality of product they 
desire, provides additional motivation to avoid disputes 
between customer and contractor. The probability of disputes 
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is reduced by adopting fully the ISO Standards, regardless of 
any arbitration clause that might be a part of the contract. 

The follow-up discussion concerning the use of 
registrars as arbitrators revealed insights into two more 
systemic problems regarding quality assurance that ISO 9000 
has the potential to resolve or alleviate. These potential 
problems include: 

(1) Adversarial attitudes. There were repeated 
comments that implementation of arbitrators would increase the 
adversarial posturing that now exists. Samples of these 
comments are paraphrased: 


• It would be a concern, just because it throws somebody 
else into the equation and gets an adversarial 
relationship going instead of the teamwork atmosphere and 
the partnership atmosphere that ought to be going between 
the customer and supplier. There's been too much of that 
in the defense portion; I saw it over at ... too much and 
there's too much of it in non-defense anyway. But it is 
getting better. 


• I think the registrar could be used between the two 
participants where they contact the registrar and get the 
registrar's point of view on it. But as an official legal 
arbitration device, I'd just hate to see that happen 
because it gets the adversarial relationship going again. 
It's us versus you. Let's call this third party to decide 
between us, and they decide, and now I'm angry because 
they decided against me, and I resent that ... 


The discussion of registrars as arb:^i-rators 
revealed just how hostile the relationship between DoD and 
defense contractors can become. The interviewees are all 
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experienced quality assurance managers whose comments show a 
sensitivity toward any idea or proposal that has the potential 
of increasing the adversarial attitudes between DoD and 
defense contractors. 

i2) Quality of Government versus ISO auditors. A 
second topic that arose frequently during discussions of 
registrars as arbitrators dealt with comparisons of ISO 
auditors to Government auditors- These comparisons surfaced 
when interviewees were discussing the advantages of using ISO 
registrars to settle quality disputes. A sampling of these 
comments is provided below: 


• I gave a paper at a symposium awhile ago talking about 

9858 and that was one of the biggest [topics] ; ... it was 

to a mixed group of military and civilian-type people ... 
the thing that I harangued on was the qualification of the 
auditors that came into ity plant. I got all sorts of 
grief, because I said that these people aren't 
professional ... they're misfits, malcontents, and 
otherwise unqualified individuals who couldn't forge a 
career for themselves in other places and they got dumped 
into the quality area. I really resent that. I resent 
those kinds of people coming into my plant and casting my 
destiny. They focus on the minutia and the wherefores and 
the whereases and they do not understand the major issues 
involved to assure the integrity of the product being 
delivered to them. 

• Right now I am more impressed with the auditor from the 
registrar than I have been from the Government folks. 

• What we find is that ISO auditors dig much deeper in 
certain areas because they are looking at the whole 
business cycle as opposed to what Government auditors do. 

• The other benefit that you can include in your thesis is 
that the third parties are also audited, whereas, no one 
audits DCAS. And also the third-party auditors are all 
certified through the European [accreditation process]. 


55 




All of the auditors that these folks use are certified by 
them. So you are coming in with some extremly heavy 
weights instead of someone that's very well versed in one 
topic. The benefit of that is that these folks will come 
in and they do an in-depth three-to-five-day survey, 
[every six months] much like you do with the QARs 
initially or every three to five years. 


The main complaint concerning Government 
auditors appears to be a lack of training and an adversarial 
attitude. It should be pointed out that the ISO auditors are 
paid by the company they are auditing and thus probably treat 
the people they are auditing as their customers. This does 
not imply that the ISO auditors aren't rigorous in their 
audits. They do have their reputation and professional 
standards to maintain, but an auditor does not have to assume 
an adversarial attitude to be effective. 

The second area of difference noted is 
directly tied to the level of training and education. The 
ISO-registering companies are sending experienced and highly 
trained professionals to conduct these audits. These 
professionals usually have technical or engineering 
backgrounds and frequently have extensive experience in the 
industry they are auditing. This level of professionalism 
makes communication with the audited company much easier; the 
company being audited is not "training" the auditor. 

The entire question of how to implement ISO 
9000 brings with it opportunities to improve the way DoD 
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administers quality assurance oversight. At the same time, 
any changes implemented in this field bring with it the risk 
of aggravating the adversarial element found in some areas of 
DoD contracting . 

4. Question Four. 

Z£ DoD were to Impose ISO 9000 Steuidards, including 
registration, what are the barriers that a company faces in 
complying with ISO 9000? 

a. Discuaaion 

The cost of registration is the obvious starting 
point when discussing the impact of imposing ISO 9000 
standards, but the researcher was also interested in 
determining if there were other barriers created when ISO 9000 
is contractually imposed. The cost to become ISO 9000 
registered varies dramatically, based on several factors 
including: 

• what registration authority is selected to perform the 
audits and registration; 

• the condition of the quality assurance program before the 
initial registration audits are performed; and 

• the size and - type of business that is applying for 
registration. 

b. Analyaia 

The registrars work on a cost-per-hour basis. If 
a company is prepared before the registrar makes the initial 
audit and review of the company's quality program, than the 
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time spent by the registrar is greatly reduced and cost of 
registration is lowered. There are numerous consulting firms 
that provide pre-registration preparation services to help 
reduce the time and expense associated with the registration 
process. Thus, it becomes difficult to determine the cost of 
registration for any given company or industry. A firm that 
has a solid quality assurance program in place will greatly 
reduce its registration expense in contrast to another company 
of comparable size and industry that is using only a limited 
program of end-item inspection. The interviewees frequently 
mentioned that a firm operating under MIL-Q-9858 is well on 
the way to ISO 9000 compliance and thus faces a lower 
registration cost. 

The researcher followed up the comments on cost 
(given registration costs were a problem) with the question: 
are there any other barriers that ISO imposes that would keep 
a company from completing registration? The answers in this 
area fell into the following four different categories. 

(1) The Scope of ISO is Too Broad. In the first 
category are those comments that describe ISO 9000 as a broad 
dociament that requires a great deal of effort on the part of 
management to interpret and customize in order to implement 
it. A sample of these is paraphrased below: 

• This ISO docviment is a short docviment that has very little 
in terms of how to do anything. What it does tell you is 
what you must do. Design is a good example; it says you 
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must state the design requirements, you must implement the 
design, demonstrating that you are complying with the 
requirements, you must verify [and] have the documentation 
to show you have satisfied the design requirements and you 
must verify independently. That's all it says; it doesn't 
tell you how to do it. 

• ... not that it is complicated, but I see companies 
struggling with it only because the standard itself is not 
articulate so that you can go and address specific areas; 
there is some interpretation involved. 

• The hardest part with ISO is to make sure that you 
interpret it in the manner that it needs to be 
interpreted. 

• The ISO 9000 in my mind is basically a blank sheet of 
paper. It is a tell-us-what-it-is-you-do. It gives 
minimal guidance in the implementation of our quality 
process. 


These comments seem to imply that 
implementation of ISO 9000 Standards requires the company 
management's commitment to review and modify its quality 
system, based on broad guidelines that require upper 
management interpretation. This indicates that ISO 9000 is 
not a cookbook approach to implementing a new quality system 
in a company. It requires the effort and time of the entire 
organization to successfully accomplish ISO registration. 

(2) Differences in Philosophy. The second common 
category of comments flows out of the interpretation issue 
raised above. Several of the people interviewed clearly drew 
a distinction between the philosophy underlying ISO 9000 
versus the normal U.S. approach to the design of a quality 
system. The interviewees all point to ISO being a step toward 
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continuous improvement and Total Quality Management. To the 
extent that ISO represents a fundamental change in the way 
companies view their quality assurance efforts, it can prove 
to be a barrier to a company's willingness to adopt ISO 
Standards. Add to this the fact that a lot of companies are 
quite satisfied with their current quality assurance systems 
and the imposition of ISO Standards could well eliminate 
potential suppliers from obtaining DoD contract awards. A few 
of the representative comments in this area are paraphrased 
below: 


• ... a lot of companies have vertical quality control and 

quality assurance. What I mean by that is the technical 
product control, if you will, within che actual 

manufacturing process and maybe even the design and 
development process, but the ISO 9000 is addressing the 
whole company. And this is what's new to U.S. businesses. 
It goes with the whole issue of continuous improvement and 
Total Quality Management. 

• Yes [there are barriers other than just cost] and it's 
primarily that some companies have quality systems in 
place that do not parallel ISO 9000. They feel their 
quality systems are very adequate and are very happy with 
them, and they are producing quality products. This might 
be especially true in the case of a larger company 
let's say Motorola or something like that where if you 
have to go in and make changes to the system it's [impact 
will be felt throughout the organization] . Whereas in a 
smaller company where you can quickly make a little 
change, it does not impact you. If you have to make a 
massive change across the board [in a large company], you 
have to communicate it to all people, put new procedures 
in place, etc. So there is that concern among those 
companies. 
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(3) Organizational Resistance to Change. The 
third category of comments is simply the age-old problem of an 
organization's resistance to change. ISO represents change to 
many of the firms implementing it. A few of the 
representative comments along these lines are paraphrased: 

• It is people's resistance to change, that is the biggest 
one. There are no drawbacks to it, other than the fact 
that it does cost you a couple of bucks. But people that 
have been doing business as usual for so long are so 
resistant to change; that is probably by far the biggest 
hurdle to overcome. 

• The only other factor, of course, is that it is new and 
they are not familiar with it, and they would have to 
learn it and learn about it. 

(4) ISO is Difficult to Use in Some Environments. 

This fourth category came from people who were responsible 

for implementing ISO in their companies. The comments dealt 
with the problem of implementing ISO in an environment that is 
not purely manufacturing. Many companies in the high 
technology fields incorporate some creative design efforts 
within their manufacturing systems -- in particular, the 
computer software efforts that are needed to support many of 
today's high tech products. These people found implementing 
ISO, where there is no clearly defined process, to be a 
challenge. Some of the representative comments follow: 
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• There are organizations that are very well structured, 
typically a manufacturing organization and somewhat a 
service organization, or even a hardware engineering 
organization, which are more accustomed to rigor and 
process. If you try to implement this in a software world 
where there is more freedom of product expression, most 
places don't have that level of rigor in their software 
environment. So implementing this process in a software 
world will require a fine balance of a process which is 
flexible enough to allow that creativity but sufficiently 
defined to allow a defined and controlled process flow. 
And that is a fine level which is, I think, not obvious to 
implement. 

• It was much easier and quicker [within our company] for 
our manufacturing/logistics organizations and our hardware 
engineering organization to come up to speed. The 
software side [still] has a long haul. 

• What you need to be able to do is define a process which 
gives an overview or the right templates of things to 
address, but doesn't make everything required for every 
implementation because of the different size of the 
projects [or] scope [or] technology. Basically you have 
to give a framework or a guideline and then leave it up to 
the individual implementation of what makes sense for that 
particular project. 


This last set of cor.urents raises the issue that 
imposing ISO in a Research and Development (R&D) type contract 
would prove to be extremely difficult. This same issue arises 
when judging the quality of R&D or software under MIL-STDS. 
The current solution to this problem is to impose MIL-I-45208 
or some other end-item delivery inspection. ISO 9001 and 9002 
are not equivalent to MIL-I-45208 because they do not stress 
end-item inspection. The underlying theme of ISO is to judge 
whether the quality system is within design parameters and if 
there is room for improvement. It is hard to make these 
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judgments when the process is not well defined or amenable to 
quantifiable measurement. 

5. Question Five 

If you were given the opportunity to tell DoD what it 
should do with regards to ISO 9000, what would you say? 

a. DiacuBBion 

This question was meant to elicit general comments 
and provide an opportunity to benefit from the collective 
experience and wisdom of this unique group of people. 

b. Analyaia 

The question yielded a wide scope of answers and 
what is paraphrased below are those comments the researcher 
felt were most pertinent to a DoD Contracting Officer. 


• I believe that unless a company is third-party accredited 
and there is some formal process of audit and review of 
the quality system, then there is no way for anybody to 
tell whether that company or group is really performing 
and adhering to that standard. You are required to do 
internal audits, but there is no way anybody can 
demonstrate that they are adhering to their own quality 
manual or quality policy statements or whatever, unless 
there is some form of quality auditing being performed. 

• I believe that from the standpoint of us competing, and 
the things that are being learned these days on how to 
manage and control products and processes, I think you 
have to rethink how we've historically done it and start 
utilizing some of these new concepts. If we don't, then 
we don't compete very well and we don't do things very 
timely. Our company has shown some dramatic changes. Even 
though we struggle in different areas, it is an ongoing 
kind of thing and just by focusing on process management 
and things of this nature, I think you see dramatic 
changes in the way companies operate. 
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• One thing that is obvious, and you are probably well aware 
of it, is that 9001 is not nearly enough. There are 
gaping holes in those standards that need to be addressed 
to have a truly competitive company. For example, nowhere 
in ISO 9001, 9002 or 9003 do you see a requirement for a 
total quality environment in a company--one that's driven 
towards quality improvement and things like that. You 
touch on it in the corrective actions sections, but there 
is nothing in there, [like] clause 3.6 of 9858 telling you 
to measure quality costs. It caused a lot of 
consternation in industry, but it's a hell of a good idea. 


• It is not only a benefit to doing a better job of managing 
our business, but it's also a vehicle for staying in the 
game with our customers. And there is nothing we did to 
achieve compliance that didn't make good business sense. 

• My experience in complying with 9858 goes back 10 years. 

There seemed to be more concentration on the form of what 
the document requires, as opposed to the substance of what 
the requirement of the docioment was. The auditors that 
came in spent all their time looking at whether a gauge 
was out of calibration. And when they found one. 

Hurrah! That was a big victory for the auditors under 
9858. That philosophy and mentality doesn't seem to 
persist with ISO. 

• We are just afraid that ISO 9000 in the next five years is 

going to lower its standards. We don't want [ISO] to 

lower their standards. But it is so backed-up like you 

wouldn't believe, with ISO requests. I just hope they 
don't lower their standards because of the workload. 

• It is one of those hard things to quantify, but I can 

judge quality, and I can figure out the cost of quality 
here. We recently acquired another company and my 

credits, as far as customer returns, have not changed. 
Yet I have doubled my line items from this building alone. 
This is absorbing inventory from three major warehouses 
and our quality has not suffered. Not to mention, we 
recently hired 30 new people and the training that we had 
put in place, has been working out fantastically. I have 
30 [new] people here who I never felt that training curve 
as far as our credits [customer returns] to line items go. 
As much as getting the ISO 9000 certification did for us, 
from a sales standpoint, it did as well for us from a 
quality standpoint. 

• [With] ISO I have a concern that it might get locked in 
concrete. That could be a problem, along with 
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interpretation of it -- the same interpretation problems 
you get in MIL-Q-9858, even with the handbook H50 
available [are inherent in ISO] . 

• It is a very positive step [to go to ISO 9000]. However, 
there is a risk that if you focus too much on process, 
then you lose some of your product metrics which is the 
traditional QA approach. My fear there is that process 
focus is good long-term business effort, but if it is 
focused just on process without tying to some direct 
product results, you can easily miss the mark and waste 
your energy. 


These general comments are intended to highlight 
potential future problems for Contracting Officers. Each 
observation sheds valuable light on such situations. And 
since these observations come from experienced ISO 9000 
quality assurance managers, they are worth noting. 


C. SUMMARY 

The telephone surveys revealed an enthusiastic and growing 
support for ISO 9000 among professional quality assurance 
managers. The current number of U.S. ISO-registered companies 
is 225. As this initial base of companies increases, the 
application of ISO 9000 Standards in DoD contracts will 
undoubtedly grow and will replace MIL-Q-9858 and MIL-I-45208. 
It is incumbent on a Contracting Officer to stay abreast of 
this development and realize the potential and actual impact 
of making this change in quality standards. 

Chapter V summarizes the data discussed and analyzed in 
this chapter, draws conclusions, supports recommendations for 
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implementation of ISO in DoD contracts, and suggests areas for 
additional research. 
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V. CONCLUSIONS, RECOMMENDATIONS AND AREAS FOR ADDITIONAL 

RESEARCH 

A. INTRODUCTION 

This study has attempted to assess the movement toward 
adoption of ISO 9000 Standards in terms of DoD contracting. 
In order to make that assessment. Chapter II discussed the new 
international ISO 9000 Standards and their equivalent MIL- 
STDS. This discussion was followed by a detailed comparison 
of the two sets of standards in Chapter III. Once this basic 
understanding of what ISO is, versus MIL-STDS, the researcher 
interviewed current users of ISO 9000 in order to solicit 
their input regarding the use of ISO Standards in Government 
contra',’ting. 

Based on both the survey performed and the comparison of 
ISO tc MIL-STDS, the researcher has developed the following 
conclusions, recommendations, answers to research questions 
and areas for additional research. 

B. CC -ICLUSIONS 

On 3 of the purposes of this study was to provide an 
assess.lent of the current movement within DoD and industry as 
to the adoption of ISO 9000 Standards. Regarding that 
assessment, the first conclusion of this study is: 
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1. The DoD policy toward ISO 9000 is not currently a 
factor in the decision of companies to become ISO 9000 
registered. 

The responses to the second question of the telephone 
survey clearly indicate that DoD quality assurance policy was 
not a factor in the decision to become ISO 9000 registered for 
those companies that have completed ISO registration. It 
should be noted that the companies contacted represent the 
leaders in the movement to ISO 9000 Standards. The motivation 
of these leaders is based on their marketing strategies for 
international sales. Each of these companies intends to 
comply with ISO 9000 in order to remove a barrier to their 
participation in international markets, and in particular, the 
European markets. 

In the event that DoD makes ISO 9000 compliance a 
contractual requirement, this conclusion would undoubtedly 
change radically. It should be remembered that this study was 
conducted during a time when ISO 9000 is still in its infancy 
in the U.S. There were only 225 companies that were ISO 9000 
registered at the time the telephone survey was conducted. 

The conclusion that current and/or contemplated DoD 
policy is not a factor in the decision to be ISO registered, 
does not preclude such policies from becoming key factors in 
ISO registration, once it becomes widely known that DoD is 
considering using ISO standards in place of applicable MIL- 
STDS. 
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The second conclusion of this study is: 

2. The adoption of ISO 9001 in place of MIL-Q-9858A will 
have no major cost impact for those companies currently 
operating under MZL-Q-9858A. 

This conclusion is drawn from the comparison of ISO 
9001 to MIL-Q-9858A, presented in chapter III, and from 
numerous comments made by people during the telephone survey. 
Several contractors remarked incidentally that they saw no 
difference between ISO 9001 and MIL-Q-9858A. A number also 
stated that the system they had developed for compliance with 
MIL-Q-9858A covered the majority of the compliance 
requirements contained in ISO 9001. The OSD(P&L) comparison 
cited in Chapter III also concluded that ISO 9001 and MIL-Q- 
9858A are fundamentally compatible as to scope and cost 
impact. 

The significance of this conclusion to Contracting 
Officers will become apparent in future meetings where 
contractors will cite ISO 9001 requirements in a contract as 
the basis of increased cost to fulfill the contract. Any 
assertion of added cost due to the use of ISO 9001, from a 
contractor who is currently conforming to MIL-Q-9858A, should 
be seriously challenged. To the extent changes to quality 
assurance procedures are required to conform to ISO 9001, such 
changes should be relatively minor in scope and cost impact. 
The philosophy of ISO 9001 is to ensure that the process 
remains under control, and emphasizes feedback that leads to 
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continuous improvement of the process. In addition, any costs 
incurred in changing quality assurance procedures should be 
part of an indirect overhead expense pool. These costs should 
be borne across all segments of a firm's business. 

If a contractor were to cite the direct costs of 
becoming ISO 9000 registered as a basis for added contract 
costs, the Contracting Officer has two responses. First, 
registration is not currently a DoD requirement. Second, if 
DoD were to make it a requirement, the company would realize 
some benefits in the registration process that offset any 
direct costs incurred. These benefits were mentioned by 
several of the persons interviewed during discussions 
concerning survey question four. The cost of registration was 
always cited as a barrier to adopting ISO 9000, but several 
contractors were quick to point out that the registration 
process helped identify weaknesses in their current quality 
assurance systems. These weaknesses had to be corrected prior 
to achieving registration; thus the registration process acted 
as a catalyst for improvements that yield future benefits. 
This correction process was considered by several quality 
assurance managers to be a significant source of improvement 
that will bear tangible savings in the near future. An 
independent third party, coming in to review a company's 
entire quality assurance system, is bound to generate new 
ideas and points of view that incumbents of a system have 
failed to recognize. The quotes cited in Chapter IV 
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concerning the perceived quality of ISO 9000 auditors adds 
weight to this line of reasoning. The perception is that ISO 
auditors are professionals with significant education and 
experience. The respondents all conveyed the feeling that ISO 
registration is a meaningful process. The companies contacted 
had paid significant sums of money to achieve registration. 
It can be reasonably concluded that those expenditures were 
subjected to the normal review and justification criteria 
inherent in organizations that are upgrading the way they do 
business. The initial impetus may have come from marketing, 
but the respondents all made a strong case for the benefits 
derived by improvements in their quality assurance 
departments, brought about by ISO registration. 

Regardless of whether a contractor cites direct or 
indirect costs, the conclusion of this researcher is that 
using ISO 9001 in place of MIL-Q-9858A is not reasonable 
justification for such additional costs. 

The third conclusion of this study is: 

3. The adoption of ISO 9002 in place of MIL-I-45208 will 
impact the cost for those cos^anies currently operating under 
MIL-I-45208. 

The comparison presented in Chapter III between ISO 
9002 and MIL-I-45208 clearly shows that ISO 9002 is 
structurally and philosophically a different standard than 
MIL-I-45208. ISO 9002 takes ISO 9001 and deletes those 
paragraphs that apply to quality control of the design 
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process. ISO 9002 still has a continuous improvement element 
and stresses control of the process. MIL-I-45208 was 
siimmarized by many respondents as an end-item acceptance 
inspection. It does not attempt to integrate the quality 
assurance function into the entire process; it simply strives 
to detect defective material after it has been produced. The 
implementation of ISO 9002 in a company operating under MIL-I- 
45208 would require the formalization of procedures, and 
documentation of the entire process, leading up to end-item 
inspection. This increased documentation and control results 
in added costs, both immediate costs to establish the systems, 
and the continuing costs of maintenance. The trade-off to 
these costs should be the realization of lower scrap and 
rework charges that arise under an end-item inspection system. 
Regardless of possible offsetting benefits, a review of the 
differences between ISO 9002 and MIL-I-45208 shows that there 
is added cost in changing from MIL-I-45208 to ISO 9002. 

The Contracting Officer who uses ISO 9002 as a 
contractual requirement must be prepared to deal with the 
added costs that will be incurred by this change to a 
contractor's quality assurance system. 

The fourth conclusion of this study is: 

4. The implementation of ISO 9000 Standards will still 
require the same type of amplifying policies and procedures 
that are currently used with MIL-STDS. 
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There is a widespread misconception among even 
knowledgeable quality assurance managers that MIL-Q-9858A and 
MIL-I-45208 are huge detailed documents more than an inch 
thick. In reality, both these MIL-STDS are brief concise 
documents that reference other standards and handbooks, 
resulting in layer upon layer of documents that detail the 
implementation of a quality assurance system commonly known as 
"MIL-Q." These layers of added definition and direction 
evolve and change in response to the problems experienced. 
The ISO standards, on the other hand, were first issued in 
1987 and there has not been enough time for them to generate 
this layering. As discussed in Chapter II, the TC176 
committee is currently working on changes and amplification of 
the basic ISO 9000 series of standards, and the layering 
effect will soon become apparent. The counterpart to many of 
these TC176 initiatives can be found in the handbooks and 
references associated with MIL-Q-9858A. 

The fifth conclusion of this study is: 

5. A requirement for ISO 9000 registration in a DoD 
contract would create reslstemce to that requirement from some 
DoD contractors. 

As pointed out by one of the respondents to the 
telephone interview, some companies are receiving the 
information and level of quality they want from their current 
quality assurance system. These companies will not willingly 
bear the costs required to achieve ISO 9000 registration until 
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the benefits of registration are better defined and 
quantified. 

The sixth conclusion of this study is: 

6. The use o£ ISO 9000 requires a different philosophy 
regarding quality assurance than that required to operate 
xmder HIL-STDS. 

The commitment of a company's top management to the 
major tenets of ISO 9000 is a requirement that must be 
satisfied with a published and actively-supported policy. 
Several of the respondents made it clear that prior to 
granting registration, ISO auditors look closely at the 
corporate management attitudes toward quality assurance. The 
ISO auditors were alsw described as "consultants" by one 
respondent, and several mentioned the impression that the ISO 
auditors were v'orking with them to improve the overall 
process, rather than simply inspecting it for compliance. 
This fostering the attitude of continuous improvement by ISO 
is one of the key differences between ISO 9000 and MIL-STDS 
and it cannot be overemphasized. ISO standards require 
feedback from the user and continual monitoring of the 
production process to ensure it is within tolerance levels. 

If ISO standards are to be used in place of MIL-STDS, 
this same change in philosophy from Government auditors would 
be necessary to ensure that maximum effectiveness is gained 
from the switch to ISO standards. Government auditors would 
have to focus their attention on aiding the contractor in 
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improving his process by identifying weak or out-of-tolerance 
areas. This approach to quality assurance is new to the 
Government, and has taken form in a program labeled In-plant 
Quality Evaluation (IQUE). IQUE embraces the idea of the 
Government Quality Assurance Representative (QAR) becoming a 
member of the contractor's team to ensure that a quality 
product is delivered to the Government. The implementation of 
ISO 9000 Standards would reinforce this philosophy and 
strengthen this new program. 

C. RECOMMENDATIONS 

The secondary objective of this study was to look at ISO 
9000 and see how it could be implemented within the DoD 
contracting arena. It is assumed that DoD will adopt ISO 9000 
Standards and it is the implementation of those standards that 
generates the following six recommendations. 

The first recommendation of this study is: 

1. DoE should define its policy of replacing MIL-Q-9858 
with ISO 9001, and begin integrating it into DoD contracts. 
This integration should be accompanied by an education progr£un 
for Contracting Officers covering the costs and benefits of 
ISO 9001, as compared to MIL-Q-9858A. 

The quality assurance side of DoD, as represented by 
OSD(P&L), has decided that ISO 9001 is an improvement over 
MIL-Q-9858A and it wants to impose these standards on DoD 
contractors. This decision is based on several factors, but 


75 




a most important element to bear in mind is that imposing ISO 
9001 on a contractor already using MIL-Q-9858A should not 
result in significant added cost. This change to ISO 9001 
promises to hold benefits in better quality products, along 
with reducing the adversarial atmosphere that is prevalent in 
some industries between QARs and contractors. By accepting 
ISO 9001, contractors are aided in achieving standardization 
of a key element in their business, thus reducing the 
redundancy that contractors experience when trying to maintain 
two systems -- one for DoD and one for commercial markets. 

To make the transition from a purely technical 
decision to implementing ISO 9001 in a contract will require 
that Contracting Officers be given a basic understanding of 
the impact this change will cause. If a contractor approaches 
a Contracting Officer about ISO 9001, it is incumbent on the 
Contracting Officer to provide a response based on at least a 
fundamental knowledge of the new system. 

The education/training is not limited to Contracting 
Officers. DoD quality assurance personnel, especially QARs, 
need to be retrained in the manner in which they approach the 
subject of quality assurance. As pointed out in conclusion 
seven, imposition of ISO 9000 on a contractor requires 
acceptance of a new philosophy, which calls for better 
communication and cooperation between quality auditors and the 
production personnel involved. 
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The recommendation to implement ISO 9001 in contracts 
leads directly to the second recommendation: 

2. DoD should train and educate their QARs in ISO 9000, 
and adopt and apply its philosophy to quality audits. 

The support of IQUE, as mentioned in conclusion seven, 
is but one example of the benefit DoD could realize by 
actively supporting the retraining of QARs to ISO 9000 
Standards. DoD has recognized that its quality operations in 
the past have not achieved the desired level of cooperation 
and support from contractors. ISO 9000 gives DoD an 
opportunity to improve both the contractor's and DoD's quality 
procedures by getting both sides to agree to use the same set 
of standards. Hopefully, both will learn to use ISO 9000 in 
a way that improves the production process, thereby improving 
quality. 

The third recommendation of this study is meant to 
support an atmosphere of cooperation that ISO 9000 makes 
possible: 

3. The idea of using ISO 9000 registrars as arbitrators 
in disputes between DoD and contractors should r t be adopted. 

This idea of finding an alternative disputes 
resoliihirn (ADR) method for settling quality issues before 
they become legal issues, has appeal at first glance. The 
problem with using a registrar as an arbitrator lies in the 
risk of reinforcing the adversarial relationship between DoD 
and contractors. ISO standards support a goal of obtaining 
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feedback from customers and using it to improve the process. 
The use of ADR instead of open communication regarding 
problems puts DoD back into an adversarial role with 
contractors. 

The introduction of registrars into dispute 
resolutions also means that a company is now faced with a new 
antagonistic relationship, forced by DoD regulation. This new 
relationship is between the company and the registrar. The 
idea behind ISO registration is to provide an experienced and 
helpful review of a quality assurance program, not to act as 
a third-party buffer between a company and its customers. 

During the telephone interviews, it became clear that 
the quality managers and directors contacted felt any disputes 
regarding quality could be settled at a lower level, rather 
than by an outside third party. These same managers asserted 
that if DoD QARs were properly trained in ISO standards, the 
poor communication between them and contractor personnel would 
be reduced, since both sides should now have the same set of 
policies and procedures to use in resolving disputes. 

This use of ISO 9000 to reduce friction is taken one 
step further in recommendation four by reducing the workload 
experienced by both the contractor and DoD personnel. The 
fourth recommendation is: 

4. Successful completion of ISO registration should be 
equivalent to a satisfactory pre-award survey of a 
contractor's quality assureuice system. 
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The ISO auditors use almost exactly the same procedure 
used during pre-award surveys by QARs and Administrative 
Contracting Officers. They review the policies and procedures 
of a company for adequacy, and then audit the quality 
assurance process to see if it is conforming to stated policy 
and procedures. DoD would better utilize its scarce resources 
by eliminating pre-award quality surveys, thereby reducing the 
workload, while gaining the benefit of experienced teams of 
highly trained professional auditors, doing the work of pre¬ 
award quality surveys at no cost to DoD. The ISO auditors 
registering companies to ISO 9000 Standards must themselves be 
audited and certified as acceptable registrars. This auditing 
and certification of registrars is currently based on European 
governmental oversight through the qualification process for 
a company to become certified as a registrar. 

The fifth recommendation of this study is: 

5. Impose ISO 9000 only in those contracts in which it 
makes sense. 

ISO 9000 standards are designed for, and heavily 
weighted toward, the creation of a quality assurance system 
for production processes. The standards are written in such 
a way that implementation is much easier where the production 
process of the end item is well defined. As stated in the 
telephone surveys, and supported by a review of the standards, 
ISO 9000 is not easily applied to situations that require 
creative production processes. 
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The comparison of ISO 9002 to MIL-I-45208 also 
demonstrates that ISO standards may not be used as exact 
replacements for MIL-STDS. Imposing ISO 9002 on a contractor 
not used to anything but end-item inspection will result in 
added cost, as explained in conclusion three. If ISO 9002 is 
to be a contractual requirement, the cost impact needs to be 
acknowledged and addressed. 

The sixth recommendation of this study is: 

6. In^lementatlon of ISO standards in DoD contracts must 
be done over a reasonable period of time. 

Any move toward contractual requirements to use ISO 
9000 Standards will take an extended period of time to phase 
in. There were only 225 companies registered to ISO 9000 at 
the time the telephone survey was made, and thus there are a 
great number of companies that have to learn about ISO 9000, 
and expend resources to implement it, before it could be taken 
as a DoD-wide requirement. 

The phase-in period must be further extended if 
recommendation four, which calls for the use of registration 
as a means to reduce pre-award quality surveys, is fully 
implemented. Since most companies are not ISO registered, 
registration costs could conceivably create a barrier to 
competition; any use of registration should be as an 
alternative to existing procedures, to allow sufficient time 
for DoD contractors to analyze and decide if registration 
costs are worth the benefits. It is not reasonable to demand 
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ISO registration when some companies may not feel the need to 
change from their existing system. In addition, registrars 
are experiencing a demand for their services to register 
companies, that exceeds their capacity to supply. 

D. ANSWERS TO RESEARCH QUESTIONS 

The primary research question of this study is: Should 
ISO 9000 be Implemented within DoD and the Defense Industrial 
Base, euid when Implemented, how will it affect DoD 
contracting? 

The answer to this question is contained in the previously 
discussed conclusions and recommendations, but can be briefly 
summarized as follows: 

1. Put ISO 9001 in place of MIL-Q-9858A in future 
contracts. 

The cost impact of complying with ISO 9001 instead of 
MIL-Q-9858A for most contractors is negligible. At the same 
time, the benefits of adopting an improvement-oriented quality 
assurance attitude are great. The use of a standard quality 
assurance policy, aimed at continuous improvement for both 
commercial and DoD work, will result in economic savings to 
the contractor and better quality products for DoD. 

2. Do not make it a contractual requirement that a 
contractor be registered to ISO standards. 

There were only 225 companies in the U.S. that were 
registered at the time this study and survey was conducted. 
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This small number of ISO registered companies means that a 
registration requirement would create barriers to competition. 

3. The primary impact of ISO 9000 will be to change the 
philosophy of DoD quality assurance progreuns from one of 
inspection to continuous improvement and prevention of poor 
quality products. 

This change in philosophy will have both cost impact 
and benefits. The Contracting Officer needs to make a 
determination that the benefits outweigh the cost before 
advocating and supporting the use of ISO 9000. This 
determination needs to be made on a contract-by-contract 
basis. Answers to secondary research questions can be found 
throughout this study; they are briefly summarized below: 

1. What is ISO 9000? 

ISO 9000 is a set of quality assurance standards 
created to provide uniformity among the European Community, 
starting in 1992. These standards stress process control and 
are a step toward continuous improvement. Chapter II, Part B, 
presents a detailed explanation of each of the ISO 9000 
Standards. 

2. What are the similarities and differences between ISO 
standards and MIL-STDS? 

Both sets of standards are process-control related, 
but ISO has a fundamentally different philosophy that stresses 
improvement and active involvement of production with quality 
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control issues. The differences between the two sets of 
standards are set forth in Chapter III of this study. 

3. What is the policy of DoD with regards to ISO 9000? 

Its stated policy is to move toward implementation of 
ISO standards in DoD contracts, but not to require 

registration. An in-depth review of DoD policy is presented 
in Chapter II, Part D, subpart 2. 

4. What are the anticipated benefits from implementing 
ISO standards? 

An improvement in the quality of products delivered 
to DoD is anticipated by changing the focus of quality 

assurance from end-item inspection to one of process control 
and feedback. The ISO standards provide DoD with the 
opportunity to assist contractors by implementing one standard 
that can be used in both commercial and defense work, with 
regards to quality assurance; it provides a step toward 

improvement of quality assurance systems. These benefits are 
evidenced in Chapter IV by the discussion and presentation of 
telephone interviews, conducted with quality managers of 
companies that have implemented ISO 9000. 

5. What steps need to be taken to implement ISO 

standards? 

Steps that should be taken in this area involve the 
following actions: 

a. Educate and train personnel in ISO 9000 standards. 
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This education process will be aimed at primarily 
DoD and contractor QARs; but contracting and program 
management personnel will also need to have a general 
understanding of these new standards. 

b. Invoke ISO 9001 in place of MIL-Q-9858A in DoD 
contracts, and provide the option of using ISO 9002 in place 
of MIL-I-45208. 

The time has come to move ISO 9000 out the realm 
of quality assurance specialists and implement it in DoD 
contracts. Comparisons and analyses discussed in Chapters II 
and III clearly demonstrate that ISO 9000 is superior to the 
current DoD practices regarding quality assurance. 

c. Assist communication with contractors as to ISO 
9000 and inform the DoD industrial base concerning its 
existence. 

There are still large segments of DoD and industry 
which are not aware of the ISO movement and what it means. In 
order to invoke fully ISO 9000, it becomes necessary to 
convert the DoD industrial base to its use. 

The entire idea of ISO 9000 is so new that it 
easily gives rise to additional areas for further research. 
A few of these areas are delineated below: 
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E. AREAS FOR ADDITIONAL RESEARCH 


Recoiranended topics for further research include: 

1. A cost-benefit analysis for implementing ISO 9001 in 
place of MIL-Q-9858A, and a separate analysis of ISO 9002 
rather than MIL-I-45208. 

2. Study of education requirements for DoD personnel 
regarding ISO, and determination of the best means of 
satisfying those requirements. 

3. Costs and benefits companies can expect from ISO 
registration. 

4. Case study of a company that has implemented ISO 
standards to determine the actual benefits of using these 
standards. 


85 








APPENDIX A 


LIST OF TELEPHONE INTERVIEWS 

1. Aeroquip Corporation - Aerospace Division, Jackson, MI. 

2. American Flange & Manufacturing Company Incorporated, Carol 
Stream, IL. 

3. Arrow Electronics Incorporated, Brookhaven, NY. 

4. Beloit Corporation - Paper Machine Division, Beloit, WI. 

5. Brand-Rex Company, Willimantic, CT. 

6. Copolymer Rubber & Chemical Company, Baton Rouge, LA. 

7. Dentsply International Incorporated, Long Island City NY 

8. Digital Equipment Corporation, Greenville, SC. 

9. DSC Communications Corporation, Plano, TX. 

10. Fischer & Porter, Warminister PA 

11. Flo-Bend Incorporated, Tulsa OK 

, 12. FMC Corporation - Fluid Control Division, Stephenville TX 

13. The Foxboro Company, E. Bridgewater MA 

14. General Electric Aircraft Engines, Lynn MA 

15. Harnischfeger Corporation - Industrial and Electric Products 
Group, Milwaukee WI 

16. Lutron Electronics, Coopersburg PA 

17. Magnetrol International Incorportated, Downers Grove IL 

18. Prime Computer Incorporated, Bedford MA 

19. RAM Electronics, Long Beach CA 

20. Reliable Automatic Sprinkler Company, Mount Vernon NY 

21. Rohrback Cosasco Systems Incorporated, Santa Fe Springs CA 

22. Rosemount .^nalytical, Orrville OH 
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23. STC Submarine Systems Incorportated, Portland OR 

24. Valmet-Appleton Incorporated, Appleton WI 

25. Yarway Corporation, Blue Bell PA 
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APPENDIX B 


COMPARISON OF MIL-Q-9858A TO ISO 9001 


MIL-Q-9858A 

ISO 9001 

No equivalency 

4 Quality system re<iuirements 


4.1 Mangement resonsibility 


4.1.1. Quality policy 


The supplier's management shall 
define and document its policy 
and objectives for, and 
commitment to, quality. The 
supplier shall ensure that this 
policy is understood, 
implemented and maintained at 
all levels in the organization. 


4.1.2 Organization 


4.1.2.1 Responsibility and 
authority 


The responsibility, authority 
and the interrelation of all 
personnel who manage, perform 
and verify work affecting 
quality shall be defined; 
particularly for personnel who 
need the organizational freedom 
and authority to: 


a) initiate action to prevent 
the occurrence of product 
nonconformity; 


b) identify and record any 
product quality problems; 


c) initiate, recommend or 
provide solutions through 
designated channels; 


d) verify the implementation 
of solutions; 
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" ’ to identify and evaluate 

problems and to 
e, recommend or provide 
ns. Management regularly 
eview the status and 
'adequacy of the quality program. 
iThe term "quality program 
[requirements" as used herein 
identifies the collective 
requirements of this 
specification. It does not mean 
that the fufillment of the 
requirements of this 
specification is the 
responsibility of any single 
contractor's organization, 
function or person. 


^o equivalency 


continuing suitability and 
effectiveness. Records of such 
reviews shall be maintained (see 
4.16). 

Note - Management reviews 
normally include assessment of 
the results of internal quality 
audits, but are carried out by, 
or on behalf of, the supplier's 
management, viz management 
personnel having direct 
responsibility for the system. 
(See 4.17) 


4.2 Quality system 

The supplier shall establish and 
maintain a documented quality 
system as a means of ensuring 
that product conforms to 
specified requirements. This 
shall include 

a) the preparation of 
documented quality system 
procedures and instructions in 
accordance with the requirements 
of this international standard; 

b) the effective 
implementation of the documented 
quality system procedures and 
instructions. 

Note - In meeting specified 
requirements, timely 
consideration needs to be given 
to the following activities: 

a) the preparation of quality 
plans and a quality manual in 
accordance with the specified 
requirements; 

b) the identification and 
acquisition of any controls. 
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rocesses, inspection equipment, 
fixtures, total production 
resources and skills that may be 
needed to achieve the required 
quality.; 

c) the updating, as necessary, 
of quality control, inspection 
and testing techniques, 
including the development of new 
instrumentation; 

d) the identification of any 
measurement requirement 
involving capability that 
exceeds the kown state of the 
art in sufficient tiine for the 
needed capability to be 
developed; 

e) the clarification of 
standards of accepatability for 
all features and requirements, 
including those which contain a 
subjective element; 

f) the compatibility of the 
design, the production process, 
installation, inspection and 
test procedures and the 
applicable documentation; 

g) the identification and 
preparation of qualtity records 
(see 4.16). 


Contract review 


The supplier shall establish and 
maintain procedures for contract 
review and for the coordination 
of these activities. 

Each constract shall be reviewed 
by the supplier to ensure that 

a) the requirements are 
adequately defined and 
documented; 












responsible for compliance with 
all provisions of the contract 
and for furnishing specified 
supplies and services which meet 
all the requirements fo the 
contract. If any inconsistency 
exists between the contract 
schedule or its general 
provisions and this 
specification, the contract 
schedule and the general 
provisions shall control. The 
contractor's quality program 
shall be planned and used in a 
manner to support reliability 
effectively. 


4.1 Drawings, Documentation an 
Changes. A procedure shall be 
maintained that concerns itself 
with the adequacy, the 
completeness and the currentness 
of drawings and with the control 
of changes in design. With 
respect to the currentness of 
drawings and changes, the 
contractor shall assure that 
requirements for the effectivity 
point of drawings and changes 
are met and that obsolete 
drawings and change requirements 
are removed from all points of 
issue and use. Some means of 
recording the effective points 
shall be employed and be 
available to the Government. 

With respect to design 
drawings and design 
specifications, a procedure 
shall be maintained that shall 
provide for the evaluation of 
the engineering adequacy and an 
evaluation of the adequacy of 
proposed changes. The 
evaluation shall encompass both 
the adequacy in relation to 
standard engineering and design 
practices and the adequacy with 
respect to the design and 
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4.4 Design Contro 

4.4.1 General 

The supplier shall establish and 
maintain procedures to control 
and verify the design of the 
product in order to ensure that 
the specified requirements are 
met. 

4.4.2 Design and development 
planning 

The supplier shall draw up plans 
that identify the responsibility 
for each design and development 
activity. The plans shall 
describe or reference these 
activities and shall be updated 
as the design evolves. 

4.4.2.1 Activity assignment 

The design and verification 
activities shall be planned and 
assigned to qualified personnel 
equipped with adequate 
resources. 

4.4.2.2 Organizational and 
technical interfaces 
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purpose of the product to which 
the drawing relates. 

With respect to supplemental 
specifications, process ins¬ 
tructions, production engineer¬ 
ing instructions, industrial 
engineering instructions and 
work instructions relating to a 
particlar design, the contractor 
shall be responsible for a 
review of thier adequacy, 
currentness and completeness. 

The quality program must provide 
complete coverage of all 
information necessary to produce 
an article in complete 
conformity with requirements of 
the design. 

The quality program shall 
assure that there is complete 
compliance with contract 
requirements for proposing, 
approving, and effecting of eng¬ 
ineering changes. The quality 
program shall provide for 
monitoring effectively the 
drawing changes of lesser 
importance not requiring 
approval by Government design 
authorities. 

Delivery of correct drawings 
and change information to the 
Government in connection with 
data acquisition shall be an 
integral part of the quality 
program. This includes full 
compliance with contract 
requirements concerning rights 
and data both proprietary and 
other. The quality program's 
responsibility for drawings and 
changes extend to the drawings 
and changes provided by the sub- 
contracors and vendors for the 
contract. 


between different groups shall 
be identified and the necessary 
information documented, 
transmitted and regularly 
reviewed. 

4.4.3 Design input 

Design input requirements relat¬ 
ing to the product shall be 
identified, documented and their 
selection reviewed by the 
supplier for adequacy. 

Incomplete, ambiguous or con¬ 
flicting requirements shall be 
resolved with those responsible 
for drawing up these 
requirements. 

4.4.4 Design output 

Design output shall be 
dociamented and expressed in 
terms of requirements, 
calculations and analyses. 

Design output shall 

a) meet the design input 
requirements; 

b) contain or reference 
acceptance criteria; 

c) conform to appropriate 
regulatory requirements whether 
or not these have been stated in 
the input information; 

d) identify those 
characteristics of the design 
that are crucial to the safe and 
proper functioning of the 
product. 

4.4.5 Design verification 

The supplier shall plan, 
estabish, document and assign to 
competent personnel functions 
for verifying the design. 
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Design verification shall 
establish that design output 
meets the design input require¬ 
ment (see 4.4.4) by means of 
design control measures such as: 

a) holding and recording 
design reviews (see 4.16); 

b) undertaJcing qualification 
tests and demonstrations; 

c) carry out alternative 
calculations; 

d) comparing new design with a 
similar proven design, if 
available. 

4.4.6 Design changes 

The supplier shall establish and 
maintain procedures for the 
identification, documentation 
and appropriate review and 
approval of all changes and 
modifications. 


Docuiment contro 

4.5.1 Document approval and 
issue 


The supplier shall establish and 
maintain procedures to control 
all documents and data that 
relate to the requirements of 
this International Standard. 
These documents shall be re¬ 
viewed and approved for adequacy 
by authorized personnel prior to 
issue. This control shall ensure 
that 

a) the pertinent issues of 
appropriate documents are 
available at all locations where 
operations essential to the 
effective functioning of the 
quality system are performed; 





b) obsolete documents are 
promptly removed from all points 
of issue or use. 


4.5.2 Document changes/ 
modifications 


. CONTROL OF PURCHASES 

5.1 Responsibility. The 

contractor is responsible for 
assuring that all supplies and 
services procured from his 
suppliers (subcontractos and 
vendors) conform to the contract 
requirements. The selection of 
sources and the nature and 
extent of control exercised by 
the contractor shall be 
dependent upon the type of sup¬ 
plies, his supplier's demon¬ 
strated capability to perform. 


Changes to documents shall be 
reviewed and approved by the 
same functions/organizations 
that performed the original 
review and approval unless 
specifically designated other¬ 
wise. The designated organiz¬ 
ations shall have access to 
pertinent background information 
upon which to base their review 
and approval. 

Where practicable, the nature of 
the change shall be identified 
in the document or the 
appropriate attachments. 

A master list or equivalent 
document control procedure shall 
be established to identify the 
current revision of documents 
inthe use of non-applicable 
documents. 

Documents shall be reissued 
after a practical number of 
changes have been made. 


4.6 Purcnasing 

4.6.1 General 

The supplier shall ensure that 
purchased product conforms to 
specified requirements. 

4.6.2 Assessment of sub¬ 
contractors 

The supplier shall select sub¬ 
contractors on the basis of 
their ability to meet sub¬ 
contract requirements, includ- 




~ and the quality evidence made 
available. To assure an adequate 
and economical control of such 
material, the contractor shall 
utilize to the fullest extent 
objective evidence of quality 
furnished by his suppliers. When 
the Government elects to perform 
inspection at a supplier's 
plant, such inspection shall not 
be used by contractors as evid¬ 
ence of effective control of 
quality by such suppliers. The 
inclusion of a product on the 
Qualified Products List only 
signifies that at one time the 
manufacturer made a product 
which met specification require¬ 
ments. It does not relieve the 
contractor of his responsibility 
for furnishing supplies that 
meet all specification require¬ 
ments or for the performance of 
specified inspections and tests 
for such material. The effect¬ 
iveness and integrity of the 
control of quality by his 
suppliers shall be assessed and 
reviewed by the contractor at 
intervals consistent with the 
complexity and quantity of 
product. Inspection of products 
upon delivery to the contractor 
shall be used for assessment and 
review to the extent necessary 
for adequate assurance of 
quality. Test reports, inspect¬ 
ion records, certificates and 
other suitable evidence relating 
to the supplier's control of 
quality should be used in the 
contractor's assessment and 
review. The contractor's 
responsibility for the control 
of purchases includes the 
establishment of a procedure for 
(1) the selection of qualified 
suppliers, (2) the transmission 
of applicable design and quality 
requirements in the Government 
contracts and associated tech- 


ing quality requirements. The 
supplier shall establish and 
maintain records of acceptable 
sub-contractors (see 4.16). 

The selection of sub¬ 
contractors, and the type and 
extent of control exercised by 
the supplier, shall be dependent 
upon the type of product and, 
where appropriate, on records of 
subcontractor's previously 
demonstrated capability and 
performance. 

The supplier shall ensure that 
quality system controls are 
effective. 
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nical requirements, (3) the 
evaluation of the adequacy of 
procured items, and (4) 
effective provisions for early 
information feedback and 
correction of nonconformances. 



Purcnasing data 


Purchasing documents shall 
tor contain data clearly describing 
s a the product ordered, including, 
rol iwhere applicable, 

a) the type, class, style, 
grade or other precise 
identification; 

b) the title or other 
positive identification, and 
applicable issue of specific¬ 
ations, drawings, process re¬ 
quirements, inspection 
instructions and other relevant 
technical data, including 
requirements for approval or 
qualification of product, 
procedures, process equipment 
and personnel; 

c) the title, number and issue 
of the quality system 
International Standard to be 
applied to the product. 

The supplier shall review and 
approve purchasing documents for 
adequacy of specified 
requirements prior to release. 

4.6.4 Verification of purchased 
product 

Where specified in the contract, 
the purchaser or his 
representative shall be afforded 
the right to verify at source or 
upon receipt that purchased 
troduct conforms to specified 
























or vendor source when such 
action is necessary to assure 
that the contractor's respon¬ 
sibility for complete assurance 
of product quality. Require¬ 
ments shall be included for 
chemical and physical testing 
and recording in connection with 
the purchase of raw materials by 
his suppliers. The purchase 
orders must also contain a 
requirement for such suppliers 
to notify and obtain approval 
from the contractor of changes 
in design of the products. 
Necessary instructions should be 
provided when provision is made 
for direct shipment from the 
sub-contractor to Government 
activities. 

7.2 Government Property. 

7.2.1 Government-furnished 
Material. 

When material is furnished by 
the Government, the contractor's 
procedures shall include at 
least the following: 

a) Examination upon receipt, 
consistent with practicability 
to detect damage in transit; 

b) Inspection for completeness 
and proper type; 

c) Periodic inspection and 
precautions to assure adequate 
storage conditions and to guard 
against damage from handling and 
deterioration during storage; 

d) Functional testing, either 
prior to or after installation, 
or both, as required by contract 
to determine satisfactory 
operation; 

e) Identification and protect¬ 
ion from improper use or dis¬ 
position; and 

f) Verification of quantity. 


requirements. Verification by 
the purchaser shall not absolve 
the supplier of the 
responsibility to provide 
acceptable product nor shall it 
preclude subsequent rejection. 
When the purchaser or his 
representative elects to carry 
out verification at the sub¬ 
contractor's plant, verification 
shall not be used by the 
supplier as evidence of effecive 
control of quality by the sub¬ 
contractor . 


4.7 Purchaser supplied product 

The supplier shall establish and; 
maintain procedures for 
verification, storage and main¬ 
tenance of purchaser supplied 
product provided for incorp¬ 
oration into the supplies. Any 
such product that is lost, 
damaged or is otherwise unsuit¬ 
able for use shall be recorded 
and reported to the purchaser 
(see 4.16). 

Note - Verification by the 
supplier does not absolve the | 
purchaser of the responsibility 
to provide acceptable product. 

found damaged, malfunctioning, 
or otherwise unsuitable for use. 
In the event of damage or mal- 
fucntioning during or after 
installation, the contractor 
shall determine and record 
probable cause and necessity for 
withholding material from use. 
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7.2.2 Damaged Government- 
furnished Material . The 

contractor shall report to the 
Government Representative 
anyGovernment-furnished material 


No equivalency 

4.8 Product identification and 
traceed^ility 

Where appropriate, the supplier 
shall establish and maintain 
procedures for identifying the 
product from applicable 
drawings, specificaitons or 
other documents, during all 
stages of production, delivery 
and installation. 

Where, and to the extent that, 
traceability is a specified 
requirement, individual product 
or batches shall have a unique 
identification. This identif¬ 
ication shall be recorded (see 
4.16) . 

6.2 Production Processing and 
Fabrication. The contractor's 
quality program must assure that 
all machining, wiring, batching, 
shaping and all basic production 
operations of any type is 
accomplished under controlled 
conditions. Controlled condit¬ 
ions include documented work 
instructions, adequate product¬ 
ion equipment, and any special 
working environment. Documented 
work instructions are considered 
to be the criteria for much of 
the production, processing and 
fabrication work. These 
instructions are the criteria 
for acceptable or unacceptable 
"workmanship”. The quality 
program will effectively monitor 
the issuance of and compliance 
with all of these work 
instructions. 

Physical examination, measure¬ 
ment or tests of the material or 

4.9 Process control 

4.9.1 General 

The supplier shall identify and 
plan the production and, where 
applicable, installation 
processes which directly affect 
quality and shall ensure that 
these processes are carried out 
under controlled conditions. 
Controlled conditions shall 
include the following: 

a) documented work 
instructions defining the manner 
of production and installation, 
where the absence of such 
instructions would adversely 
affect quality, use of suitable 
production and installation 
equipment, suitable working 
environment, compliance with 
reference standards/codes and 
quality plans; 
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products processed is necessary 
for each work operation and must 
also be conducted under 
controlled conditions. If 
physical inspection of processed 
manterial is impossible or 
disadvantageous, indirect 
control by monitoring processing 
methods, equipment and personnel 
shall be provided. Both physical 
inspection and process 
monitoring shall be provided 
when control is inadequate 
without both, or when contract 
or specification requires both. 

Inspection and monitoring of 
processed material or produces 
shall be accomplished in any 
suitable manner selected by the 
contractor. Methods of 
inspection and monitoring shall 
be corrected any time their 
unsuitability with reasonable 
evidence is demonstrated. 
Adherence to selected methods 
for inspection and monitoring 
shall be complete and continous. 
Corrective measures shall be 
takentaken when noncompliance 
occurs. 

Inspection by machine 
operators, automated inspection 
gages, moving line or lot 
sampling, setup or first piece 
approval, production line 
inspection station, inspection 
or test department, roving 
inspectors - any other type of 
inspection - shall be employed 
in any combination desired by 
the contractor which will 
adequately and efficiently 
protect product quality and the 
integrity of processing. 

Criteri for approval and 
rejection shall be provided for 
all inspection of product and 
monitoring of methods, equip¬ 
ment, and personnel. Means for 


b) monitoring and control of 
suitable process and product 
characteristics during 
production and installation; 

c) the approval of processes 
and equipment, as appropriate; 

d) criteria for workmanship 
which shall be stipulated, to 
the greatest practicable extent, 
in written standards by means of 
representative samples. 


4.9.2 Special processes 

These are processes, the results 
of which cannot be fully 
verified by subsequent 
inspection and testing of the 
product and where, for example, 
processing deficiencies may 
become apparent only after the 
product is in use. Accordingly, 
continuous monitoring and/or 
compliance with documented 
procedures is required to ensure 
that che specified requirements 
are met. These processes shall 
be qualified and shall also 
comply wiLh requirements of 
4.9.1. 

Records shall be maintained for 
qualified processes, equipment 
and personnel, as appropriate. 
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Inspection and testing 

4.10.1 Receiving inspection and 
testing 

4.10.1.1 The supplier shall 
ensure that incoming product is 
not used or processed (except in 
the circumstances described in 
4.10.2) until it has been 
inspected or otherwise verified 
as conforming to specified 
requirements. Verification shall 
be in accordance with the 
quality plan or doc\amented 
procedures. 

4.10.1.2 Where incoming product 
is released for urgent product¬ 
ion purposes, it shall be pos¬ 
itively identified and recorded 
(see 4.16) in order to permit 
immediate recall and replace¬ 
ment in the event of noncon¬ 
formance to specified require¬ 
ments . 

Note - In determining the amount 
and nature of receiving 
inspection, consideration should 
be given to the control 
exercised at source and doc¬ 
umented evidence of quality 
conformance provided. 
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Excerpts from 6.2 Production 
Processing emd Fabrication 

I Physical examination, measure- 
pnent or tests of the material or 
products processed is necessary 
for each work operation and must 
also be conducted under 
controlled conditions. If 
hysical inspection of processed 
aterial is impossible or disad- 
antageous, indirect control by 
onitoring porcessing methods, 
equipment and personnel shall be 
provided. Both physical inspect¬ 
ion and process monitoring shall 
be provided when control is 
inadequate without both, or when 
contract or specification 
requires both. 

Inspection by machine 
operators, automated inspection 
gages, moving line or lot 
sampling, setup or first piece 
approval, production line 
inspection station, inspection 
or test department, roving 
inspectors - any other type of 
inspection - shall be employed 
in any combination desired by 
the contractor which will 
adequately and efficiently 
protect product quality and the 
integrity of processing. 

Criteria for approval and 
rejection shall be provided for 
all inspection of product and 
monitoring of methods, equip¬ 
ment, and personnel. Means for 
identifying approved and 
rejected product shall be 
rovided. 


.3 Completed Item Inspection 
and Testing. The quality program 
shall assure that there is a 
system for final inspection and 
test of completed products. Such 
testing shall provide a measure 
of the overall quality of the 
completed product and shall be 


4.10.2 In-process inspection an 
testing 

The supplier shall 

a) inspect, test and identify 
product as required by the 
quality plan or documented 
procedures; 

I 

b) establish product conform¬ 
ance to specified requirements 
by use of process monitoring and 
control methods; 

c) hold product until the 
required inspection and tests 
have been completed or nec¬ 
essary reports have been re¬ 
ceived and verified except when 
product is released under pos¬ 
itive recall procedures (see 
4.10.1). Release under positive 
recall procedures shall not 
preclude the activities outlined 
in 4.10.2a); 

d) identify nonconforming 
product. 
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performed so that it simulates, 
to a sufficient degree, product 
end use and functioning. Such 
simulation frequently involves 
appropriate life and endurance 
tests and qualification testing. 
Final inspection and testing 
shall provide for reporting to 
designers any unusual difficul¬ 
ties, deficiencies or question¬ 
able conditions. When modific¬ 
ations, repairs or replacements 
are required after final in¬ 
spection or testing, there shall 
be reinspection and retesting of 
any characteristics affected. 


ied either on receipt of pro¬ 
duct or in-process, have been 
carried out and that the data 
meets specified requirements. 

The supplier shall carry out all 
final inspection and testing in 
accordance with the quality plan 
or dociamented procedures to 
complete the evidence of 
conformance of the finished 
product to the spec-ified 
requirements. 

No product shall be despatched 
until all the activities spec¬ 
ified in the quality plan or 
documented procedures have been 
satisfactorily completed and the 
associated data and doc¬ 
umentation is available and 
authorized. 


4.10.4 Inspection and test 
records 



the supplier shall establish and 
maintain records which give 
evidence that the product has 
passed inspection and/or test 
with defined acceptance criteria 
(see 4.16). 


4.11 Inspection, measuring and 
test equipment 

The supplier shall control, 
calibrate and maintain inspect¬ 
ion, measuring and test equip¬ 
ment, whether owned by the sup¬ 
plier, on loan, or provided by 
the purchaser, to demonstrate 
the conformance of product to 
the specified requirements. 
Equipment shall be used in a 
manner which ensures that meas¬ 
urement uncertainty is known and 
is consistent with the required 
measurement cap-ability. 
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conformity with military spec- 
ification MIL-C-45662. In add¬ 
ition, the contractor shall 
insure the use of only such sub¬ 
contractor and vendor sources 
that depend upon calibration 
systems which effectively 
control the accuracy of 
measuring and testing equipment. 


The supplier shall 

a) identify the measurements 
to be made, the accuracy 
required and select the approp¬ 
riate inspection, measuring and 
test equipment; 

b) identify, calibrate and 
adjust all inspection, meas¬ 
uring and test equipment and 
devices that can affect product 
quality at prescribed inter¬ 
vals, or prior to use, against 
certified equipment having a 
known valid relationship to 
nationally recognized standards 
- where no such standards exist, 
the basis used for cal-ibration 
shall be documented; 

c) establish, document and 
maintain calibration proced¬ 
ures, including details of 
equipment type, identification 
number, location, frequency of 
checks, check method, accept¬ 
ance criteria and the action to 
be taken when results are un¬ 
satisfactory; 

d) ensure that the inspect¬ 
ion, measuring and test equip¬ 
ment is capable of the accuracy 
and precision necessary; 

e) identify inspection, 
meas-uring and test equipment 
with a suitable indicator or 
approved identification record 
to show the calibration status; 

f) maintain calibration 
records for inspection, meas¬ 
uring and test equipment (see 
4.16); 

g) assess and document the 
validity of previous inspect-ion 
and test results when in¬ 
spection, measuring and test 
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h) ensure that the environ- 
ental conditions are suitable 

for the calibration, inspect¬ 
ions, measurements and tests 
being carried out; 

i) insure that the handling, 
preservation and storage of 
inspection, measuring and test 
equipment is such that the 
accuracy and fitness for use is 
maintained; 

j) safeguard inspection, 
measuring and test facilities, 
including both test hardware and 
test software, from adjust-ments 
which would invalidate the 
calibration setting. 





rw*'! 


Inspection and test status 


The inspection and test status 
of product shall be identified 
by using markings, authorized 
stamps, tags, labels, routing 
cards, inspection records, test 
software, physical location or 
other suitable means, which 
indicate the conformance or 
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Corrective Action. The 

quality program shall detect 
promplty and correct assignable 
conditions adverse to quality. 
Design, purchasing, manufactur¬ 
ing, testing or other operations 
which could result in or have 
resulted in defective supplies, 
services, facilities, technical 
data, standards or other 
elements of contract performance 
which could create excessive 
losses or costs must be identi¬ 
fied and changed as a result of 
the quality program. Corrective 
action will extend to the per¬ 
formance of all suppliers and 
vendors and will be responsive 
to data and product forwarded 
from users. Corrective action 
shall include as a minimiim: 

a) Analysis of data and exam¬ 
ination of product scrapped or 
reworked to determine extent and 
causes; 

b) Analysis of trends in pro¬ 
cesses or performance of work to 

revent nonconforming product; 


c) re-graded for alternative 
applications, or 

d) rejected or scrapped. 

Where required by the contract, 
the proposed use or repair of 
product (see 4.13.1b) which does 
not conform to specified 
requirements shall be reported 
for concession to the purchaser 
or his representative. The 
description of nonconformity 
that has been accepted, and of 
repairs, shall be recorded to 
denote the actual condition (see 
4.16) . 

Repaired and reworked product 
shall be re-inspected in 
accordance with documented 
rocedures. 


.14 Corrective action 

The supplier shall establish, 
document and maintain procedures 
for 

a) investigating the cause of 
nonconforming product and the 
corrective action needed to 
prevent recurrence; 

b) analysing all processes, 
work operations, concessions, 
quality records, service re¬ 
ports and customer complaints to 
detect and eliminate potent-ial 
causes of nonconforming product; 

c) initiating preventive 
actions to deal with problems to 
a level corresponding to the 
risks encountered; 

d) applying controls to ensure 
that corrective actions are 
taken and that they are ef¬ 
fective; 
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(jues 


Where appropriate, the supplier 
shall establish procedures for 
identifying adequate statist¬ 
ical techniques required for 
verifying the acceptability of 
process capability and product 
characteristic 
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subject to review by the 
cognizant Government Represent¬ 
ative. Any sampling plan used 
shall provide valid confidence 
and quality levels. 

No equival-ancy 




Internal quality audits 


The supplier shall carry out a 
comprehensive system of planned 
and documented internal quality 
audits to verify whether quality 
activities comply with planned 
arrangements and to determine 
the effectiveness of the quality 
system. 

Audits shall be scheduled on the 
basis of the status and 
importance of the activity. 

The audits and follow-up actions 
shall be carried out in 
accordance with documented 
procedures. 

The results of the audits shall 
be documented and brought to the 
attention of the personnel 
having responsibility in the 
area audited. The management 
personnel responsible for the 
area sahll take timely 
corrective action on the 
deficiencies found by the audit 
(see 4.1.3) . 
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3.6 costs Rt^atM to Quality. |No equivalency 
The contractor shall maintain 
jand use quality cost data as a 
panagement element of the 
jquality program. These data 
shall serve the purpose of id- 
lentifying the cost of both the 
prevention and correction of 
nonconforming supplies (e.g., 
laobr and material involved in 
material spoilage caused by 
defective work, correction of 
defective work and for quality 
control exercised by the con¬ 
tractor at subcontractor's or 
(vendor's facilities). The 
specified quality cost data to 
be maintained and used will be 
determined by the contractor. 

These data shall, on request, be 
identified and made available 
for "on site" review by the 
(Government Representative. 

.4 Use of Contractor's Inspect- (No equivalency 
ion Equipment. The contracrtor's 
gages, measuring and testing 
[devices shall be made available 
for use by the Government when 
required to determine conform- 
jance with contract requirements. 

If conditions warrant, contract¬ 
or's personnel shall be made 
available for operation of such 
devices and for verification of 
their accuracy and condition. 
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7.1 Ooverzuaent Inspection at 
Subcontractor of Vendor 
Facilities. The Governments 
reserves the right to inspect at 
source supplies or services not 
manufactured or performed with 
the contractor's facility. 
Government inspection shall not 
constitute accepance; nor shall 
it in any way replace contractor 
inspection or otherwise relieve 
the contractor of his respons¬ 
ibility to furnish an acceptable 
end item. The purpose of this 
inspection is to assist the 
Government Representative at the 
contractor's facility to deter¬ 
mine the conformance of supplies 
or services with contract re¬ 
quirements. Such inspection can 
only be requested by or under 
authorization of the Government 
Representative. When Government 
inspection is required, the 
contractor shall add to his 
purchasing document the 
following statement: 

"Government inspection is 
required prior to ship¬ 
ment from your plant. 

Upon receipt of this or¬ 
der, promptly notify the 
Government Representative 
who normally services 
your plant so that ap¬ 
propriate planning for 
Government inspection 
can be accomplished." 

When, under authorization from 
the Government Representative, 
copies of the purchasing | 

document are to be furnished 
directly by the subcontractor or 
vendor to the Government Rep- * 
resentative at his facility 
rather than through Government 
Channels, the contractor shall 
add to his purchasing document a 
statement substantially as 
follows: 
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“On receipt of this order, 
promptly furnish a copy to 
the Government representative 
who normally services your 
plant, or, if none, to the 
nearest Anny, Navy, Air 
Force, or Defense Supply Ag¬ 
ency inspection office. In 
the event the representative 
or office cannot be located, 
our purchasing agent should 
be notified immediately." 

All documents and referenced 
data for purchases applying to a 
Government contract shall be 
available for review by the Gov¬ 
ernment Representative to deter¬ 
mine compliance with the re¬ 
quirements for the control of 
such purchases. Copies of pur¬ 
chasing documents for Government 
purposes shall be furnished in 
accordance with the instructions 
of the Government Representat¬ 
ive. The contractor shall make 
available to the Government 
Representative reports of any 
nonconformance found on Govern¬ 
ment source inspected supplies 
and shall (when requested) 
require the supplier to 
coordinate with his Government 
Representative on corrective 
action. 
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APPENDIX C 


COMPARISON OF MIL-I-45208 TO ISO 9002 


MIL-I-45208 

1.1 Scope. This specification 
establihes requirements for 
contractors' inspection systems. 
These requirements pertain to 
the inspections and tests nec¬ 
essary to substantiate product 
conformance to drawings,spec¬ 
ifications and contract require¬ 
ments and to all inspections and 
tests required by the contract. 
These requirements are in add¬ 
ition to those inspections and 
tests set forth in applicable 
specifications and other con¬ 
tractual documents. 


ISO 9002 

4 Quality system re<iuirements 

4.1 Mangement responsibility 

4.1.1. Quality policy 

The supplier's management shall 
define and document its policy 
and objectives for, and 
commitment to, quality. The 
supplier shall ensure that this 
policy is understood, 
implemented and maintained at 
all levels in the organization. 
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contract. The Government at its 
option may furnish written of 


e) control further processing, 
delivery or installation of 


the acceptability or nonaccept- nonconforming product until the 


ability of the inspection 
system. The contractor shall 
notify the Government Represent¬ 
ative in writing of any change 
to his inspection system. The 
inspection system shall be sub¬ 
ject to disapproval if changes 
thereto would result in non- 
conforming product. 




deficiency or unsatisfactory 
condition has been corrected. 





The quality sys 
satisfy the req 
International S 


i adopted to 
ements of this 
■dard shall be 
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No equivalency 


reviewed at appropriate 
intervals by the supplier's 
management to ensure its 
continuing suitability and 
effectiveness. Records of such 
reviews shall be maintained (see 
4.16). 

Note - Management reviews 
normally include assessment of 
the results of internal quality 
audits, but are carried out by, 
or on behalf of, the supplier's 
management, viz management 
personnel having direct 
responsibility for the system. 
(See 4.17) 

4.2 Quality system 

The supplier shall establish and 
maintain a documented quality 
system as a means of ensuring 
that product conforms to 
specified requirements. This 
shall include 

a) the preparation of 
documented quality system 
procedures and instructions in 
accordance with the requirements 
of this international standard; 

b) the effective 
implementation of the documented 
quality system procedures and 
instructions. 

Note - In meeting specified 
requirements, timely 
consideration needs to be given 
to the following activities: 

a) the preparation of quality 
plans and a quality manual in 
accordance with the specified 
requirements; 

b) the identification and 
acquisition of any controls, 
processes, inspection equipment. 
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fixtures, total production 
resources and skills that may be 
heeded to achieve the required 
quality.; 

c) the updating, as necessary, 
of quality control, inspection 
and testing techniques, 
including the development of new 
instrumentation; 

d) the identification of any 
measurement requirement 
involving capability that 
exceeds the kown state of the 
art in sufficient time for the 
needed capability to be 
developed; 

e) the clarification of 
standards of accepatability for 
all features and requirements, 
including those which contain a 
subjective element; 

f) the compatibility of the 
design, the production process, 
installation, inspection and 
test procedures and the 
applicable documentation; 

g) the identification and 
preparation of qualtity records 
(see 4.16). 


Relation to Other Contract 
Requirements. The inspection 
system requirements set forth in 
this specification shall be 
satisfied in addition to all 
detail requirements contained in 
the statement of work or in 
other parts of the contract. The 
contractor is responsible for 
compliance with all provisions 
of the contract and for 
furnishing specified articles 
which meet all the requirements 
of the contract. To the extent 
of any inconsistency between the 
contract schedule or its general 


.3 Contract review 

The supplier shall establish and 
maintain procedures for contract 
review and for the coordination 
of these activities. 

Each constract shall be reviewed 
by the supplier to ensure that 

a) the requirements are 
adequately defined and 
documented; 
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o equivalency 


Document c 
modifications 

Changes to docioments shall be 
reviewed and approved by the 
same functions/organizations 
that performed the original 
review and approval unless 
specifically designated other¬ 
wise. The designated organiz¬ 
ations shall have access to 
pertinent background information 
upon which to base their review 
and approval. 

Where practicable, the nature of 
the change shall be identified 
in the document or the 
appropriate attachments. 

A master list or equivalent 
document control procedure shall 
be established to identify the 
current revision of documents in 
order to preclude the use of 
non-applicable documents. 

Documents shall be reissued 
after a practical number of 
changes have been made. 


Purchasing 


4.5.1 General 

The supplier shall ensure that 
purchased product conforms to 
specified requirements. 

4.5.2 Assessment of sub¬ 
contractors 


The supplier shall select sub¬ 
contractors on the basis of 
their ability to meet sub¬ 
contract requirements, includ¬ 
ing quality requirements. The 
supplier shall establish and 
maintain records of acceptable 
subcontractors (see 4.16). 

The selection of sub¬ 
contractors, and the type and 
extent of control exercised by 








the supplier, shall be dependent 
upon the type of product and, 
where appropriate, on records of 
subcontractor's previously 
demonstrated capability and 
performance. 


3.11.2 Purchasing Doc\ 2 ments~ 

When, under authorization of the 
Government Representative, 
copies of the purchasing 
document are to be furnished 
directly by the subcontractor or 
vendor to the Government Repre¬ 
sentative at his facility rather 
than through Government chan- 
els, the contractor shall add to 
his purchasing document a state¬ 
ment substantially as follows: 
"On receipt of this order, 
promptly furnish a copy to 
the Government Represent¬ 
ative who normally serv¬ 
ices your plant or, if 
none, to the nearest Army 
Navy, Air Force, or Def¬ 
ense Supply Agency inspec¬ 
tion office. In the event 
the representative or of¬ 
fice cannot be located, our 
purchasing agent should be 
notified immediately." 


The supplier shall ensure that 
quality system controls are 
effective. 

4.5.3 Purchasing data 

Purchasing documents shall 
contain data clearly describing 
the product ordered, including, 
where applicable, 

a) the type, class, style, 
grade or other precise 
identification; 

b) the title or other 
positive identification, and 
applicable issue of specific¬ 
ations, drawings, process re¬ 
quirements, inspection 
instructions and other relevant 
technical data, including 
requirements for approval or 
qualification of product, 
procedures, process equipment 
and personnel; 

c) the title, number and issue 
of the quality system 
International Standard to be 
applied to the product. 


3.11.3 Reference Data. All 

documents and referenced data 
for purchases applying to a 
Government contract shall be 
available for review by the 
Government Representative to 
determine compliance with the 
requirements for the control of 
such purchases. Copies of pur¬ 
chasing documents required for 
Government inspection purposes 
shall be furnished in accord¬ 
ance with the instructions of 
the Government Representative. 


The supplier shall review and 
approve purchasing documents for 
adequacy of specified 
requirement.: prior to release. 
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d) Functional testing, either 
prior to or after installation, 
or both, as required by contract 
to determine satisfactory 
operation; 

e) Identification and protect¬ 
ion from improper use or dis¬ 
position; and 

f) Verifir? ion of quantity. 

7.2.2 Damaged Government- 
furnished Material . The 

contractor shall report to the 
Government Representative any 
Government-furnished material 
found damaged, malfunctioning, 
or otherwise unsuitable for use. 
In the event of damage or mal- 
fucntioning during or after 
installation, the contractor 
shall determine and record 
probable cause and necessity for 
withholding material from use. 





Product 
tracaablllty 


cation an 


.4 Process Controls. Process 
control procedures shall be an 
integral part of the inspection 
system when such inspections are 
a part of the specification or 
the contract. 


Where appropriate, the supplier 
shall establish and maintain 
procedures for identifying the 
product from applicable 
drawings, specificaitons or 
other documents, during all 
stages of production, delivery 
and installation. 

Where, and to the extent that, 
traceability is a specified 
requirement, individual product 
or batches shall have a unique 
identification. This identif¬ 
ication shall be recorded (see 
4.16) . 


Process contro 
4.8.1 General 


The supplier shall identify and 
plan the production and, where 






applicable, installation 
rocesses which directly affect 
quality and shall ensure that 
these processes are carried out 
under controlled conditions. 
Controlled conditions shall 
include the following: 

a) documented work 
instructions defining the manner 
of production and installation, 
where the absence of such 
instructions would adversely 
affect quality, use of suitable 
production and installation 
equipment, suitable working 
environment, compliance with 
reference standards/codes and 
quality plans; 

b) monitoring and control of 
suitable process and product 
characteristics during 
production and installation; 

c) the approval of processes 
and equipment, as appropriate; 

d) criteria for workmanship 
which shall be stipulated, to 
the greatest practicable extent, 
in written standards by means of 
representative samples. 


4.8.2 Special processes 

These are processes, the results 
of which cannot be fully 
verified by subsequent 
inspection and testing of the 
product and where, for example, 
processing deficiencies may 
become apparent only after the 
product is in use. Accordingly, 
continuous monitoring and/or 
compliance with documented 
procedures is required to ensure 
that the specified requirements 
are met. These processes shall 
be qualified and shall also 
comply with requirements of 
4.9.1. 

























4.9.3 Final inspection and 
testing 

The quality plan or documented 
procedures for final inspection 
and testing shall require that 
all specified inspection and 
tests, including those specif¬ 
ied either on receipt of pro¬ 
duct or in-process, have been 
carried out and that the data 
meets specified requirements. 

The supplier shall carry out all 
final inspection and testing in 
accordance with the quality plan 
or documented procedures to 
complete the evidence of 
conformance of the finished 
product to the specified 
requirements. 

No product shall be despatched 
until all the activities spec¬ 
ified in the quality plan or 
documented procedures have been 
satisfactorily completed and the 
associated data and doc¬ 
umentation is available and 
authorized. 























4.9.4 Inspection and test 
records 


3.3 Measuring and Test Equip- 
ment. The contractor shall 
provide and maintain gages and 
other measuring and testing 
devices necessary to assure that 
suppliers conform to technical 
requirements. In order to assure 
continued accuracy, these 
devices shall be calibrated at 
established intervals against 
certified standards which have 
kown valid relationships to 
national standards. If product¬ 
ion tooling, such as jigs, 
fixtures, templates, and 
patterns is used as a media of 
inspection, such devices shall 
also be proved for accuracy at 
established intervals. 
Calibration of inspection equip¬ 
ment shall be in accordance with 
MIL-C-45662. When required, the 
contractor's measuring and test¬ 
ing equipment shall be made 
available for use by the 
Government Representative to 
determine conformance of product 
with contract requirerents. In 
addition, if condicions warrant, 
contractor's personnel shall be 
made available for operation of 
such devices and for verifi¬ 
cation of their accuracy and 
condition. 


The supplier shall establish and 
maintain records which give 
evidence that the product has 
passed inspection and/or test 
with defined acceptance criteria 
(see 4.16) . 

4.l6 Inspection, measitrinsr and 
test equipment 

The supplier shall control, 
calibrate and maintain inspect¬ 
ion, measuring and test equip¬ 
ment, whether owned by the sup¬ 
plier, on loan, or provi'^ed by 
the purchaser, to demonstrate 
the conformance of product to 
the specified requirements. 
Equipment shall be used in a 
manner which ensures that meas¬ 
urement uncertainty is known and 
is consistent with the required 
measurement capability. 

The supplier shall 

a) identify the measurements 
to be made, the accuracy 
required and select the approp¬ 
riate inspection, measuring and 
test equipment; 

b) identify, calibrate and 
adjust all inspection, meas¬ 
uring and test equipment and 
devices that can affect product 
quality at prescribed inter¬ 
vals, or prior to use, against 
certified equipment having a 
known valid relationship to 
nationally recognized standards 
where no such standards exist, 
the basis used for calibration 
shall be documented; 

c) establish, document and 
maintain calibration proced¬ 
ures, including details cf 
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equipment type, identification 
number, location, frequency of 
checks, check method, accept¬ 
ance criteria and the action to 
be taken when results are un¬ 
satisfactory; 

d) ensure that the inspect¬ 
ion, measuring and test equip¬ 
ment is capable of the accuracy 
and precision necessary; 

e) identify inspection, 
meas-uring and test equipment 
with a suitable indicator or 
approved identification record 
to show the calibration status; 

f) maintain calibration 
records for inspection, meas¬ 
uring and test equipment (see 
4.16); 

i 

g) assess and document the 
validity of previous inspection 
and test results when in¬ 
spection, measuring and test 
equipment is found to be out of 
calibration; 

h) ensure that the environ¬ 
mental conditions are suitable 
for the calibration, inspect¬ 
ions, measurements and tests 
being carried out; 

i) insure that the handling, 
preservation and storage of 
inspection, measuring and test 
equipment is such that the 
accuracy and fitness for use is 
maintained; 

j) safeguard inspection, 
measuring and test facilities, 
including both test hardware and 
test software, from adjustments 
which would invalidate the 
calibration setting. 
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^.*7 Nonconforming Material. The 

contractor shall establish and 
maintain an effective and 
positive system for controlling 
nonconforming material, includ¬ 
ing procedures for the identi¬ 
fication, segregation, 
presentation and disposition of 
reworked or repaired supplies. 
Repair of nonconforming supplies 
shall be in accordance with 
documented procedures acceptable 
to the Government. The 
acceptance of nonconforming 
supplies is a prerogative of and 
shall be as prescribed by the 
Government. All nonconforming 
supplies shall be positively 
identified to prevent 
unauthorized use, shipment and 
intermingling with conforming 
supplies. Holding areas, 
mutually agreeable to the 
contractor and the Government 
Representative shall be provided 
by the contractor. 


i.ii Control of nonconforming 
product 

The supplier shall establish and 
maintain procedures to ensure 
that product that does not 
conform to specified 
requirements is prevented from 
inadvertent use or installat¬ 
ion. Control shall provide for 
identification, documentation, 
evluation, segregation (when 
practical), disposition of 
nonconforming product and for 
notification to the functions 
concerned. 

4.12.1 Nonconfomity review and 
disposition 

The responsibility for review 
and authority for the 
disposition of nonconforming 
product shall be defined. 

Nonconforming product shall be 
reviewed in accordance with 
documented procedures. It may be 

a) reworked to meet the 
specified requirements, or 

b) be accepted with or with¬ 
out repair by concession, or 

c) regraded for alternative 
applications, or 

d) rejected or scrapped. 

Where required by the contract, 
the proposed use or repair of 
product (see 4.13.1b) which does 
not conform to specified 
requirements shall be reported 
for concession to the purchaser 
or his representative. The 
description of nonconformity 
that has been accepted, and of 
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repairs, shall be recorded to 
denote the actual condition, 
(see 4.16). 


Repaired and reworked product 
shall be reinspected in 
accordance with documented 
procedures. 

'4.Id Corrective action 

The supplier shall establish, 
document and maintain procedures 
for 

a) investigating the cause of 
nonconforming product and the 
corrective action needed to 
prevent recurrence; 

b) analysing all processes, 
work operations, concessions, 
quality records, service re¬ 
ports and customer complaints to 
detect and eliminate potential 
causes of nonconforming product; 

c) initiating preventive 
actions to deal with problems to 
a level corresponding to the 
risks encountered; 

d) applying controls to ensure 
that corrective actions are 
taken and that they are ef¬ 
fective; 

e) implementing and recording 
changes in procedures resulting 
from corrective action. 

4.l4 Handling^ storage, 
packaging and delivery 

4.14.1 General 

The supplier shall establish, 
document and maintain proced¬ 
ures for handling, storage, 
packaging and delivey of 
product. 








4.14.2 Handling 

The supplier shall provide 
ethods and means of handling 
that prevent damage or deter¬ 
ioration. 

4.14.3 Storage 

The supplier shall provide sec¬ 
ure storage areas or stock rooms 
to prevent damage or det¬ 
erioration of product, pending 
use or delivery. Appropriate 
methods for authorizing receipt 
and the despatch to and from 
such areas shall be stipulated. 
In order to detect deteriorat¬ 
ion, the condition of product in 
stock shall be assessed at 
appropriate intervals. 

4.14.4 Packaging 

The supplier shall control 
packing, preservation and mark¬ 
ing processes (including mater¬ 
ials used) to the extent nec¬ 
essary to ensure conformance to 
specified requirements and shall 
identify, preserve and segregate 
all product from the time of 
receipt until the supplier's 
responsibility ceases. 

4.14.5 Delivery 

The suppli '!r shall arrange for 
the protection of the quality of 
product after final inspect-ion 
and test. Where con¬ 
tractually specified, this 
protection shall be extended to 
include delivery to destination. 


.2.2 Records. The contractor 
shall maintain adequate records 
of all inspections and tests. 
The records shall indicate the 
nature and number of observ- 


.15 Quality Records 

The supplier shall establish and 
maintain procedures for id¬ 
entification, collection. 




'ations made, the number and type 
iof deficiencies found, the 
quantities approved and rejected 
and the nature of corrective 
action taken as appropriate. 

3.2.3 Corrective Action. The 

contractor shall take prompt 
action to correct assignable 
conditions which have resulted 
or could result in the 
submission to the Government of 
supplies and services which do 
not conform to (1) the quality 
assurance provisions of the item 
specification, (2) inspections 
and tests required by the 
contract, and (3) other 
inspections and tests required 
to substantiate product 
conformance. 

3.2.4 Drawings and Changes. The 

contractor's inspection system 
shall provide for procedures 
which will assure that the 
latest applicalbe drawings, 
specifications and instructions 
required by the contract, as 
well as authorized changes 
thereto, are used for 
fabrication, inspection and 
testing. 


o equivalency 


indexing, filing, storage, 
maintenance and disposition of 
quality records. 

Quality records shall be main¬ 
tained to demonstrate achieve¬ 
ment of the required quality and 
the effective operation of the 
quality system. Pertinent sub¬ 
contractor quality records shall 
be an element of these data. 

All quality records shall be 
legible and identifiable to the 
product involved. Quality 
records shall be stored and 
maintained in such a way that 
they are readily retrievable in 
facilities that provide a suit¬ 
able environment to minimize 
deterioration or damage and to 
prevent loss. Retention times of 
quality records shall be 
established and recorded. Where 
agreed contractually, quality 
records shall be made available 
for evaluation by the purchaser 
or his representative for an 
agreed period. 


4.17 Training 

The supplier shall establish and 
maintain procedures for 
identifying the training needs 
and provide for the training of 
all personnel activities 
affecting quality during 
production and installation. 
Personnel performing specific 
assigned tasks shall be 
qualified on the basis of 
appropriate education, training 
and/or experience, as required. 
Appropriate records of training 
shall be maintained (see 4.15). 
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4.17 Internal quality audits 

The supplier shall carry out a 
comprehensive system of planned 
and documented internal quality 
audits to verify whether quality 
activities comply with planned 
arrangements and to determine 
;the effectiveness of the quality 
system. 

Audits shall be scheduled on the 
basis of the status and 
importance of the activity. 

The audits and follow-up actions 
shall be carried out in 
accordance with documented 
procedures. 

The results of the audits shall 
be documented and brought to the 
attention of the personnel 
having responsibility in the 
area audited. The management 
personnel responsible for the 
area sahll take timely 
corrective action on the 
deficiencies found by the audit 
(see 4.1.3). 


o equivalency 
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No equivalency 


3.11.1 Ooverziinent Inspection 
Requirements. When Government 
inspection is required, the 
contractor shall add to his pur¬ 
chasing document the following 
statement: 

"Government inspection is 
required prior to ship¬ 
ment from your plant. Upon 
receipt of this order, 
promptly notify the Gov¬ 
ernment Representative who 
normally services your plant 
so that appropriate planning 
for Government inspection can 
be accomplished." 
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APPENDIX D 


MIL-Q-9858A/ISO MATRIX 


MIL-Q-9858 

9001 

9002 

9003 

9004 

1.1, 1.2 

N/A 

N/A 

N/A 

1 

1.3 

4.2 

4.2 

4.2 

5 

2 

N/A 

N/A 

N/A 

N/A 

3.1 

4.1.2.1 

4.1.2.1 

4.1.2.1 

4, 5.5, 5.2.3 

3.2 

4.3 

4.3 

N/A 

5.3.3, 8.3, 8.25 

3.3 

4.9.1 

4.8.1 

N/A 

11.3, 17.2 

3.4 

4.16 

4.15 

4.4, 4.6 

5.3.4, 17.3 

3.5 

4.14 

4.13 

N/A 

15 

3.6 

N/A 

N/A 

N/A 

4.3.2, 6 

4.1 

4.5 

4.4 

4.3 

17.2 

4.2 

4.11 

4.10 

4.6 

13 

4.3 

4.11 

4.10 

N/A 

N/A 

4.4 

4.11 

4.10 

N/A 

N/A 

4.5 

4.2 

4.2 

N/A 

N/A 

5.1 

4.6 

4.5 

N/A 

9 

5.2 

4.6.3 

4.5.3 

N/A 

9.5 

6.1 

4.10 

4.9 

N/A 

11.2, 12.1 

6.2 

4.9 

4.8 

N/A 

10, 12.2 

6.3 

4.10.3 

4.9.3 

4.5 

12.3 

6.4 

4.15 

4.14 

4.9 

16 

6.5 

4.13 

4.12 

4.8 

14 

6.6 

4.20 

4.18 

4.12 

20 

6.7 

4.12 

4.11 

4.7 

11.7 

7.1 

N/A 

N/A 

N/A 

N/A 

7.2 

4.7 

4.6 

N/A 

N/A 


8 N/A N/A N/A 1 
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